Time-Inconsistent Optimal Control Problems 
and the Equilibrium HJB Equation* 



Jiongmin Yong 

Department of Mathematics, University of Central Florida, Orlando, FL 32816, USA 

March 2, 2013 



Abstract 

A general time-inconsistent optimal control problem is considered for stochastic differential equations 
with deterministic coefficients. Under suitable conditions, a Hamilton- Jacobi-Bellman type equation is 
derived for the equilibrium value function of the problem. Well-posedness and some properties of such 
an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the 
linear-quadratic problem and a generalized Merton's portfolio problem are investigated. 
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1 Introduction 

Let T ^ F, P) be a complete filtered probability space on which a c?-dimensional standard Brownian motion 
W{-) is defined, whose natural filtration is F = {^t}t>o (augmented by all the P-null sets). Let T > 0. For 
any t G [0,T), we consider the following controlled stochastic different equation (SDE, for short): 

( dX{s) = b{s,X{s),u{s))ds + a{s,X{s),u{s))dW{s), s e [t,T], 

where b : [0,T] x R" x [/ ^ R" and a : [Q,T] x R" x ^> R"""^ are suitable deterministic maps with U 
being a metric space. In the above, (t, x) E [0, T) x R" is called an initial pair, u : [t,T] x ^ U is called a 
control process and X : [0,T] x Q ^ R" is called a state process. We define the set of all admissible controls 
by the following: 

U[t,T] ~ : [t,T] X n ^ U \ u(-) is F-progressively measurablej. (1.2) 

Under some mild conditions, for any {t,x) G [0,T) x R" and u(-) G U[t,T], (1.1) admits a unique solution 
X{-) = X{- ■,t,x,u{-)). To measure the performance of the control process u(-) £ U[t,T], we introduce the 
following cost functional 

J°(t,a;;M(-)) =eJ / e-^("-*)5°(s, X(s), u(s))ds -f e-^(^-*)/i°(X(r))l , (1.3) 
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Department of Applied Mathematics, Polytechnic University, Hong Kong. The author would like to thank Professors X. Li and 
J. Huang for their hospitaUty. 
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with some constant S > (called the discount rate), some deterministic maps g° : [0,T] x R" x J7 ^> IR 
and /i° : IR" — >■ IR, and Et[-] = E[- \J-'t]- On the right hand side of (1.3), the first term is referred to as a 
running cost and the second term is referred to as a terminal cost. We can pose the following optimal control 
problem. 



Problem (C). For any {t,x) e [0,T) x R", find a u(-) e W[i,T] such that 

J°(t, x; u(-)) = mi J°(t, x; u(-)) = V°(t, x). 



(1.4) 



Any u{-) G U[t,T] satisfying (1.4) is called an optimal control of Problem (C) for the initial pair {t,x), 
the corresponding state process X{-) = X{- ;t,x,u{-)) and the pair {X{-),u{-)) are called the corresponding 
optimal state process and optimal pair, respectively. The function V"°(- , •) defined by (1.4) is called the value 
function of Problem (C). 

For Problem (C), we have the following Bellman's principle of optimality ([8, 29]): For any r € [t,T], 

(1.5) 



V°{t,x)= inf eJ / e-^('*-*)/(s,X(s),u(s))ds + e-^(^-*V°(r,X(r;t,x,w(-))) 



where U[t,T] is defined similar to that oiU[t,T] replacing [t,T] by [t,T] (see (1.2)). Now, if {X{-),u{-)) is an 
optimal pair of Problem (C) for the initial pair {t,x) £ [0,T) x R", then from (1.5), we obtain 



= E( 



V°it,x) = J°{t,x;ui-)) 

e-^(^-*)gO(s, X{s), u{s))ds + g-^f^-*) j" (r, X(r; t, x, u{-)); u{-) \ j^_^j) 

j^^ e-^^'-*^g°{s, X{s),u{s))ds + e-*(^-*)y° (r, X{t; t, x, u{-))j 

e-^(*-*)<7°(s, X{s),u{s))ds + e-*(^-*V°(r, X(r; x, u(-)))l = V^°(t, x). 



(1.6) 



> inf Et 

u(.)eW[t,r] 

Thus, one must have 



E. 



j'{T,X{ry,u{.)\^^^^^)-V%T,X{r)) 



a.s. 



Since 

it follows that 



a.s. 



jO(r,X(r);u(-)|[,_^j)-F°(r,X(r))>0, 
J^r,Xiry,ui-)\ ) = V%T,X{r)) = inf J°(r, X(t); «(•)) , 



a.s. 



(1.7) 



This means that the restriction u(-)|j^yj e ZY[t, T] of an optimal control u{-) G h{[t,T] for the initial pair 
{t,x) on any later time interval [r, T] is optimal for the initial pair (r, X(r; t, a;, «(•))) . Such a phenomenon 
is called the time- consistency of Problem (C). 

The advantage of the time-consistency is that for any given initial pair {t,x), if an optimal control ■u(-) 
can be constructed for that (initial pair), then it will stay optimal hereafter (for the later initial pair along the 
optimal trajectory). This is very ideal. However, in reality, the time-consistency could be lost. An interesting 
situation that we are going to discuss in this paper is what we call the general discounting situation which 
includes the so-called non- exponential discounting, or hyperbolic discounting situations. This amounts to 
saying the following: Due to the possible subjectivity of people's preferences, the discount factors e~'''*~*^ 
and e~*^^~*^ appeared in (1.3) might be replaced by some general functions X{s,t) and v{T,t), or more 
generally, we will consider the following cost functional: 



J{t,x;u{-))='Et 



^ g{t,s,X{s), 



u{s))ds + h{t,X{T)) 



(1.8) 



where the maps g{-) and h(-) exphcitly depend on the initial time t in some general way. The optimal control 
problem associated with (1.1) and (1.8), called Problem (N), will not be time-consistent, or time-inconsistent, 
in general, meaning that a restriction of an optimal control for a specific initial pair on a later time interval 
might not be optimal for that corresponding initial pair. Some concrete examples will be presented in the 
next section. 

In recent years, time-inconsistent optimal control problems have attracted a number of researchers. See 

[5, 6, 2, 27, 19, 28, 7, 3, 13] and references cited therein. 

The purpose of this paper is to obtain time-consistent optimal controls (which should be more properly 
called equilibrium control) for Problem (N) mentioned above. Let us now briefly describe our approach. 
Inspired by [27, 28], we introduce a sequence of multi-person hierarchical difl[erential games as follows. For 
any A'' > 1, let 11 be a partition of the time interval [0, T] defined by 

n : = ti < t2 < ■ ■ ■ < tN = T, 

with the mesh size ||n|| given by 

||n|| = max (ife - tk-i). 

l<k<N 

The diflferential game, denoted by Problem (G^), associated with partition 11 consists of N players. The 

k-th player controls the system on [tfe-i, tfc) by taking his/her control u'^(-) G U[tk-i,tk]- The cost functional 
is constructed in a sophisticated way, by using some techniques of forward-backward stochastic differential 
equations (FBSDEs, for short) found in [16, 17]. The interaction among the players are as follows: (i) The 
terminal pair {tk,X{tk)) of Player k is the initial pair of Player {k + 1); (ii) All the player know that each 
player tries to find an optimal control for his/her own problem; and (iii) Each player will discount the future 
costs in his/her own way, regardless of the fact that the later players will control the system. Under certain 
conditions, each player will have an optimal control, denoted by u'^(-) G U[tk-i,tk], for his/her own problem, 
as well as his/her own value function V''{- , •) defined on [tk-i,tk] x R". Define 

N 

u''{t) = Y,n''mt.-^,t.){t), te[0,T), (1.9) 

and 

AT 

yn(t,x) = 5]v''=(i,x)/[(,_,,,,)(i), {t,x) G [o,r) X R". (1.10) 
fe=i 

We may call w^(-) and V^{- , •) the Nash equilibrium control and Nash equilibrium value function of Problem 
(G^), respectively. When the following limits 

' nrlrn^^W t G [0,T], 

liniKo 

lim V^(t, x) = V(t, x), (t, x) G [0, T] X R", 

l|n|Ko 

exist for some u(-) G U[0, T] and V : [0, T] x R" — R, we call them a time- consistent equilibrium control and 
a time- consistent equilibrium value function of Problem (N), respectively. As a major contribution of this 
paper, we have derived the equilibrium Hamilton- J acobi- Bellman equation which can be used to characterize 
the equilibrium value function V{- , •), and it recovers the result for time-inconsistent deterministic linear- 
quadratic problem presented in [27]. The well-posedness of such an HJB equation will be established for 
the case that the diffusion of the state equation does not contain the control. The general case is open at 
the moment, and we expect to present some more complete results in our future publications. As important 
and interesting special cases, we will construct equilibrium controls for stochastic LQ problem with general 
discounting and for generalized Merton's portfolio problem. 

We refer the readers to [23, 22, 10, 20, 24, 15, 14, 21, 4, 12, 11, 1], for some relevant results. 
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2 Two Examples of Time-Inconsistent Optimal Control Problems 



In this section, we present two interesting examples of optimal control problems which are time-inconsistent. 
Example 2.1. Consider a one-dimensional controlled linear SDE: 



dX{s) = u{s)ds + aX{s)dW{s), 
X{t) = X, 



s&[t,T], 



with cost functional 



J{t,x;u{-)) = 'Et / \u{s)\Hs + g{t)\X{T){^ 



(2.1) 



(2.2) 



where cr > is a constant and g : [0,T] — > (0, oo) is a deterministic non-constant, continuous function. For 
a fixed initial pair (i,x) G [0,r) x IR, the LQ problem associated with the above (2.1)-(2.2) is a standard 
one. We can show that (see Appendix) the problem admits a unique optimal pair {X{-), u{-)) given by 



u{s) 



a2+5(i)(e'-'(r-*) -1) 



e 2 



[W^(s)-lV(t)], 



se[t,T]. 



(2.3) 



Now, let T e {t,T). We consider the corresponding LQ problem starting from the initial pair (r, X(r)). 
Then the optimal pair, denoted by {X{-),u{-)), is given by 



u{s) = - 



(72+p(T)(e'^'(^-^)-l) 

a^+g(T)(e"'(^-^)-l) + .g(t)(e"'(^-") - 1) 
a2 + 5(t)(e'^'(^-^) - 1) ' 0-2 + .g(r)(e'-'(T-^) - 1) 
a2.g(T)e"'(^-^) 



Xis), 



s€[t,T]. 



(2.4) 



a2+.g(T)(e'^'('r-^)-l) 

g{T) (72+.g(/)(,.-''-'--'' -1 



9(i) a2+g(r)(e-^(^-)-l) 



u{s), 



Hence, in general 



X{s) = X{s), S(s)=7i(,s), sG[T,r], 

may fail. In fact, we have the following comparison of the cost functional values corresponding to u(-)| 
and u(-), with the same initial pair (r, X(r)): 



J 



/ ^ ,M ^ ^^(e-'^^--) - l)e-'(^--)[g(i) - g(r)]2|X(T)|2 



>o, 



[(t2 + g{t){e'^'iT-t) - l)]2[f^2 + g(r)(e<-'(r-^) - 1)] 
unless gij) = g{t) or a; = 0. Therefore, the LQ problem associated with (2.1) (2.2) is time-inconsistent. 

Note that by sending — >• 0, using rHopital's rule, we recover the example presented in [27, 28] for a 
deterministic LQ problem. 

Example 2.2. (Merton's portfolio problem) We consider a one-dimensional controlled SDE: 



dX{s) = [rX{s) + (/u - r)u(s) - c(s)] ds + au{s)dW{s), 
X{t) = X, 
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€ [t,T], 



(2.5) 



where (u(-), c(-)) is the control process with u(-) being the dollar amount invested in the stock (which could 
be positive and negative), c(-) is the consumption rate process (which has to be non- negative) , and the 
solution X{-) = X{- ■,t,x,u{-),c{-)) of (2.5) corresponding to (t, x, tt(-), c(-)) is the wealth process, which is 
required to be non-negative. In the above, r > is the interest rate for the bank account, /i > r and a are 
the appreciation rate and volatility of the stock, respectively. Let 

l{[t,T] = < u : [t, T] X — >• IR I u{-) is F-progressively measurable, E / \u{s)\'^ds < oo>, 

C[t,T] = {c: [t,T] X [0, oo) I c(-) is F-progressively measurable, E / c{s)ds < cx)|. 

The payoff functional is given by the following: 

J{t, x; u{-), c(-)) = [ ^ Ht, s)c{sfds + p{t)X{Tf] , (2.6) 

where i^(- , •) and p{-) are given positive- valued functions, and (3 € (0, 1). As a convention, we define 

= -00, X < 0. (2.7) 

The following is referred to as a generalized Merton's portfolio problem: 

Problem (M). For any given (t,x) G [0,r) x R, find a pair (m(-),c(-)) e l{[t,T] x C[t,T] such that 

J{t,x;u{-),c{-))= sup J{t,x;u{-),c{-)). (2.8) 

{u{-),c{-))eU[t,T]xC[t,T] 



When 



u{t, s) = e-*(^-*) , Pit) = e-*(^-*) , 0<t<s<T, 



(2.9) 



the above problem is called a classical Merton's portfolio problem (which is time-consistent). We now look 
at the above general case. For a fixed t G [0, T), Problem (M) can be regarded as a standard optimal control 

problem. Therefore, we can treat it in a standard way. We can show that (see Appendix) the problem admits 
a unique optimal control (u*(-), c*(-)) for the initial pair {t,x), which is given by the following: 



(/x-r)X*(s) 
<72(l-/3) ' 



u\s) 
c\s) 



s&[t,T], 



with 



[2m^(l-,i) + (/i-,f]/3 



and 



V\t,x)= sup 

(«(■), c(-))eW[t,T]xC[t,T] 



2^2(1 - (3) 
J{t,x;u{-),c{-)) 



r'^ 1 1-/3 

eT^(T-t)p(i)T^ + jf eT^(^-*V(t,T)T^dT x^ , (t, x) € [0, T] X [0, oo). 

Now, let t G {t,T), and consider Problem (M) starting from the initial pair (<, X*(f)). Then 



(2.10) 



(2.11) 



(2.12) 



V*{i,X*{t))=[e^^^-^p{t)^ + J_ e^^^-^u{t,T)T^dT\' ^X*{t}^ 



We can show that (see Appendix) if t € {t, T) is such that 



Pit) 



Pit) 



dr. 



(2.13) 



then 



j{i,X\t}-,ui-)\ ci-)\ )<V'{t,X\t})= sup J(f,X*(t);n(-),c(-)), (2.14) 

{u(-),c(-))eU[i,T]xC[i,T] 



which means that the restriction (u(-)|j^yj, c(-)|j^ j,j) of optimal control (u(-),c(-)) for the initial pair {t,x) 
on [t,T] is not optimal for the initial pair {t,X*{t)) if (2.13) holds. 

Interestingly, in the case that (2.9) holds, one has 

p{t) e-«(^-*) 

Thus, in this case, (2.13) does not hold. As a matter of fact, the problem is time-consistent and (2.13) should 
not be true. 



3 Some Preliminaries 



For convenience, let us rewrite the state equation and the cost functional below. 



dX{s) = b{s,X{s),u{s))ds + (j{s,X{s),u{s))dW{s), sG [t,T], 
X{t) = X, 



(3.1) 



and 

i-T 



J{t,x;u{-))='Et[J^ g{t,s,X{s),u{s))ds + h{t,X{T))'\. (3.2) 

Clearly, our cost functional covers the non-exponential/hyperbolic discounting situations. By comparing the 
above state equation and cost functional, it seems that we may consider a little more general state equation 
of the following form: 

I dX{s) = b{t, X, s, X{s),u{s))ds + (7{t, x, s,X{s), u{s))dW{s), s G [t, T], 
\ X{t) = X. 

However, according to [28] (see also [27]), we know that when an equilibrium pair (see below for definition) 
is constructed, the eventual effective state equation will take the following form: 

1 dX{s) = b{s,X{s),s,X{s),u{s))ds + a{s,X{s),s,X{s),u{s))dW{s), s € [t,T], 
I Xit) = X, 

Therefore, it suffices to consider the state equation of form (3.1). 

In what follows, we let T > be a fixed time horizon, and U C R™ be a closed subset, which could be 
either bounded or unbounded (it is allowed that U — R™). We will use if > as a generic constant which 
can be different from line to line. Let 5" be the set of all (n x n) symmetric real matrices. Denote 



D[Q,T] = |(i,s) G [0,r]2 I < i < s < t|. 



6 



Recall from Section 1 that 

U[t,T] = |u : [t, T] X f2 — )• [/■ I u(-) is F-progressively measurablej. 
Further, for q > 1, let 

l(''[t,T] = : [t,T] X Q ^ U \ u(-) is F-progressively measurable, \u{s)fds < ooj. 

Note that in the case U is bounded, for different g > 1, all the W^lt, T] are the same as U[t, T]. We introduce 
the following standing assumptions. 

(HI) The maps & : [0, T] x R" x ?7 R", cr : [0, T] x R" x J7 -)■ R"""^ are continuous and there exist 
constants L > and A; > such that 

\b{t,x,u) - b{t,y,u)\ < L{1 + {\x\ V \y\)'' + \u\)\x - y\, 

{x-yXt,x,u)-h{t,y,u)) <L\x-y\^, V(i,u) e [0,T] x [/, a;,?/ G R", (3.5) 

_ \a{t,x,u)-(T{t,y,u)\<L\x-y\, 

where |a;| V \y\ = max{|a;|, |?;|}, and 

\b{t,Q,u)\ + \(j{t,Q,u)\<L{l + \u\), V(t,w) G [0,r] X [/. (3.6) 



(H2) Maps g : D[0, T] x R" x J7 R and /i : [0, T] x R" -)■ R are continuous, and there exist constants 
L > and q>0 such that 

I 0<9ir,t,x,u)<L(l + \x\^ + \u\^), V(r, M, -) ^ D[0, T] x R" x C/. (3.7) 

t 0<h{T,x)<L{l + \x\'J), 



Let us make a couple of remarks on (HI). First of all, if x ^ b{t, x, u) is uniformly Lipschitz, then the first 
two conditions of (3.5) hold. On the other hand, we point out that the first condition in (3.5) merely implies 
that X b{t,x,u) is locally Lipschitz, and the second condition in (3.5) alone does not imply the global 
Lipschitz condition for the map x i-^ b{t, x,u). A simple example that the first and the second conditions in 
(3.5) are satisfied but x i-> b{t, x, u) is not uniform Lipschitz is the following: 

b{t, X, u) = -\x\^x - \u\x, {t, X, u) G [0, T] X R" X U. 

It is clear that the above map is not uniformly Lipschitz in x, the first condition in (3.5) holds with k = 2, 
and we can check that 

{x-y, bit, x, u) - b{t, y,u)) < 0, V(i, u) G [0, T] x U, x,y € R". 

Note that under (3.5), one has 

( X, b{t, x,u)) = {x, b{t, X, u) — b{t, 0,u)) + {x, b{t, 0, u) ) 

< L\xf + L{1 + \u\)\x\ < 2L{1 + \xf + \uf), 

and 

\a{t,x,u)\^ < L^{1 + \x\ + \u\)^ < 3X^(1 + \x\'' + 

Now, for (H2), we note that the nonnegativity of g{-) and h{-) can be replaced by the condition that both 
g{-) and h{-) are bounded from below. The following result is concerning the well-posedness of the state 
equation. 
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Proposition 3.1. Let (HI) hold. Then for any {t,x) £ [0,T] x R" and u{-) G W[t,T], with q>2, state 
equation (3.1) admits a unique solution X{-) = X{- ■,t,x,u{-j), and the following estimate hold: 

^X{s■,t,x,u{■))\'l <K{l + \x\'i + \u{r)\idr), s € [t,T]. (3.8) 

Moreover, if x G R" is another point, the following holds: 

W]\X{s;t,x,u{-)))-X{s;t,x,u{-))\'^ <K\x-x\'^, Vi G [0,T], a;,£ e R", u{-)eU%T]. (3.9) 

Proof. Let {t,x) G [0,T) x R" be given and u{-) G W[t,T]. By the local Lipschitz condition, we see that 
the solution X{-) = X{- ■,t,x,u{-)) exists locally, say on [t, r) for some stopping time r. Next, applying Ito's 
formula to |-^(-)|' on [f, r), we have the following: 

E|X(s)r = \^\'+^[ {Q\Xir)r^ {Xir),bir,Xir),uir))) 

+fc^|X(r)|«-4(X(r),a(r,X(r),«(r)))2 + ||X(r)|''-2tr [a{s,X{s),u{s))(T{s,X{s),u{s)r])dr 

<\x\'J+KE f |X(r)|«-2(l + |X(r)|2 + |M(r)|2)dr < |a;|« + ifE / (l + |X(r)|« + |M(r)|«)(ir. 
Jt Jt 

By Gronwall's inequality, we obtain 

E,\X{s)\i <K{l + \x\i + E.J^ \u{r)\'idr), sG[t,r). 

This implies that X{-) must be globally exists on [t,T], and estimate (3.8) holds. 

Next, for any t G [0,T), x,x G R", and u(-) G W[t,T], let X(-) = X(- ; t, a;, u(-)) and X{-) = 
X{- ■,t,x,u{-)). Then applying Ito's formula to \X{-) — X(-)p, we get 



E|X(s) -X(s)|2 = \x-x\''+-Ej (^{X{s)-X{s),h{s,X{s),u{.s))-b{s,X{s),uis))) 

+ \a{s,X{s),u{s)) - a{s, X{s), u(s))p) ds 
<\x-x\'^ +KE \X{s)-X{s)\'^ds. 

Hence, applying Gronwall's inequality, we obtain (3.9). □ 
From the above result, we see that for any {t,x) G [0,r) x R" and u(-) G W^^^[f,T], the following holds: 

E|5(r, s, X{s), u{s))\ <L{l + E|X(s)|«^2 + E|«(s)|«^2) 

< K{1 + ^E|u(s)|«^2 ^ ^ E|M(r)|«^2^r), 

and ^ 

E|/i(t,X(T))| < L(l +E|X(T)|«^2^ < + + E |u(s)|«^2^s). 

Hence, J(t, a;;u(-)) is finite for any u{-) G W'^'^[t,T]. For simplicity, hereafter, we adopt the following 
convention: 

J{t, x: u{-)) = +00, if J{t, X, u{- j) is not finite or not defined. (3.10) 
We now formally state our optimal control problem. 
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Problem (N). For any given initial pair {t,x) G [0,T) x R", find a u{-) G U[t,T] such that 

J(t,x;u(-)) = inf J(t,x;u(-)). (3.11) 

From the examples presented in the previous section, we know that the above Problem (N) is time- 
inconsistent, in general. Our goal is to find time-consistent equilibrium controls and characterize the equi- 
librium value function, which will be made precise below. 

We denote 

a{t, X, u) = -a{t, X, u)a{t, x, uf , V(t, x, u) G [0, T] x R" x U. 

Define 

H(r,i,a;,u,p,P) = {h{t,x,u),p) +tv [a{t,x,u)P] +g{T,t,x,u), 

V(t, i, a;, P) G £>[0, T] X R" X Z7 X R" X <S", 

and let 

H{T,t,x,p,P) = M Wl{T,t,x,u,p,P), V(T,t,x,p,P) G D[0,T] X R" X R" X 5". (3.13) 

Note that H{T,t,x,p, P) is not necessarily finite on the whole space £)[0, T] x R" x R" x .S" when U is 
unbounded. Therefore, we denote 

V{H) = \^{T,t,x,p,P) gD[0,T] xR" xR" x<S" I H{T,t,x,p,P) > -oo}, (3.14) 

call it the domain of H. Next, let 

argminH(T,i,x, •,]3,P) = \ ueU\ B.{T,t,x,u,p,P) = mmB.{T,t,x,u,p, P)], 

L ueu ) (3.15) 

y{T,t,x,p,P)&V{H). 

This is a multi-valued map. Suppose we can define a map V' : 2?(V') ^ ^{H) — > U such that 

H{t, t, x,p, P) = ]H(t, X, ^^{t, t, x,p, P),p, P) = inf ]H(t, t, x, u,p, P) > —oo, 



y{T,t,x,p,P)€V{iP). 



(3.16) 



The set ^^{tp) is called the domain of tp, which consists of all points {T,t,x,p, P) G T>{H) such that the 
infimum in (3.16) is achieved at tp{T,t,x,p,P). It is clear that ^{■) is actually a selection of argmin]H(-), 
i.e., 

V'(T,t,a;,jj,P) G argminH(T,i,a;,-,p,P), y{T,t,x,p,P) gV{iI)). (3.17) 

The map ip{-) will play an important role later. Therefore, let us say a little bit more on it. Note that when 
U is bounded (since it is assumed to be closed, it is compact in this case), one has 

D(^) = V{H) = D[0,T] X R" X R" X 5". 

However, when U is unbounded, say, U = R™, one might have 

^ V{H) ^ Li[0, T] X R" X R" X 5". (3.18) 

To say something about the case of (3.18), let us present the following simple lemma. 

Lemma 3.2. Let U C R™ be closed and R be a lower semi-continuous map such that 

inf f{u) = f> -00. 



For any e > 0, let 

r{u) = f{u)+e\u\^, uGU. 



Then there exists a u'^ G U such that 



Moreover, 



u£(7 



limP = /, lime|«^P = 0. 

£4,0 e4,0 



Proof. First of all, fix a wq G U. For any minimizing sequence Uk & U of f'^{-), we may assume that 
riuo) > r{uk) = f{uk)+s\ukt > I+e\uk\\ Vfc > 1. 

Thus, Uk is bounded. Consequently, by the closeness of U, we may assume that Uk ^ & U exists, which 
attains the infimum of f^{-). Next, it is clear that /^(•) decreases as s decreases, and 

/<limf = /0. 

e->-0 

Now, for any S > 0, there exists a, ug G U such that 

f{us)<f + S. 

On the other hand, 

r = fiu') < r{us) = f{us) + e\us\''. 

Hence, letting e — > 0, we get 

/° < f + s. 

Since 5 > is arbitrary, we must have f = P- Finally, by 

/ < f{u') < f{u') + e\u'\^ = r{u') = r ^ /, 

we see that 

limelu^P = 0, 

proving our conclusion. □ 
Now, for the case 'D(V') 7^ we may introduce 



^(r, t, X, u,p, P) = ]H(r, t, x, u,p, P) + e\u\^, 

{T,t,x,u,p,P) e D[0,T] X R" X [/ X R" X 5". 

Then by Lemma 3.2, one can find a ijj'^ : 'D{H) — ^ U such that 

W{T,t,x,r{T,t,x,p,P),p,P) = inf ff(r,i,a;,u,p,P) 

= H%T,t,X,p,P), {T,t,X,p,P)€V{H). 

Moreover, e H'^{T,t,x,p,P) is decreasing as £ 4-0, and 

lim H%T,t,x,p,P) = H{T,t,x,p,P), 

{T,t,x,p,P)eV{H). 

lime\i;%T,t,x,p,P)\^ = 0, 

e— 

In general, we do not expect the convergence of tp'^ir, t, x,p, P) as £ ^ 0. 
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(3.19) 



(3.20) 



Wc point out that in general the map : ©(V') C D[0,T] x R" x R" x 5" — >■ J7 is not necessarily 
continuous. Here is a simple example. 

Example 3.3. Let n = m = d=l,U = [0, 1], and 

b{t,x,u) = u, a{t,x,u) = a{t,x), g{T,t,x,u) = R{T,t)u, 
with a{- , ■) and R{- , •) being continuous, and R{t, t) > for all (r, t) G D[0, T]. Then 

M{T,t,x,u,p,P) =pu+^a{t,xyP + R{T,t)u= ^a{t,xyP+ [p + R{T,t)]u. 

Consequently, 

H{T,t,x,p,P) = ^a{t,xfP + inm{p + R{T,t),0}, {T,t,x,p,P) e D[0,T] x R" x R" x 5", 

which is continuous, and 

^{r,t,x,p,P) = /(p+fl(.,t)<o), V{r,t,x,p,P) e D[0,T] x R^ 

which has discontinuity along p + R{t, t) = 0. 

The following example shows that sometimes, ij) can also be continuous. 
Example 3.4. Let n = m = d=l, [/= [—1, 1], and 

b{t,x,u)=u, a{t,x,u) = a{t,x), g{T,t,x,u) = ^R{T,t)u'^, 

with a{- , •) and R{- , •) being continuous, and R{t, t) > for all (t, t) € D[0, T]. Then 

H(r,t,a;,u,p,P) = + ^cr(t,x)2p + ii?(T,i)u^ 

= l.«,xfP+ifl(M)(»+^)'-5j|;^. 

Consequently, 
and 

V(T,f,a;,p,P) = -[sgnp](-J^Al), V(T,i,a;,p,P) e £»[0,T] x R^. 

Clearly, both if and tjj are continuous. Also, we see that even if all the coefficients are very smooth, we 
cannot guarantee that H and tp are as smooth as the coefficients, in general. 

Ifere is another example which shows that H and tjj could be as smooth as the coefficients. 

Exemiple 3.5. Let n = m = d=l,U = (—1, 1), and 

b{t,x,u) = u, a{t,x,u) = a{t,x), g{T,t,x,u) = —R{T,t, x)ln{l — u'^), 

with R{t, t, x) being positive- valued and bounded. Then 

B.{T,t,x,u,p,P) =pu+ ^a{t,xfP - R{T,t, x)ln{l - u^). 
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A direct computation shows that 

H{t, t, x,p, P) = pi:{T, t, x,p, P) + ^a{t, xfP - R{t, t, x) In [l - V(t, t, x,p, P)^] 
- ' ait, xfP + Rir, t, X) In ^^^^ ^' + x/^^(^^WTF 



with 



2 ' R{T,t,x) + ^R{T,t,xy + '[P " ' ' 2R{T,t,x) 

ll){T,t,X,p,P) = 



R{t, t, X) + y/R{T,t,x)^ +p2 

Therefore, both H and ip are as smooth as the eoefficients. 

From the above discussion, we see that the situation concerning the map is very comphcated. For 
the simpUcity of presentation below, we adopt the following assumption. 

(H3) The map V' : £»[0, T] x R" x R" x 5" ^> [/ is well-defined and has needed regularity. 

We will address more general situations concerning in our future publications. 

Now, let us recall a standard verification theorem for Problem (C) stated in Section 1, which will be used 
below. For our later purposes, it sufiices to consider Problem (C) with the discount rate ^ = 0. We will 
denote 

Ci'2([0,T] xR") = {<,.) eC([0,T] xR") | vt{- ,-),v.{- ,-),v..{- & C{[0,T] x R")}. 

The proof of the following result can be found in [8]. 

Proposition 3.6. Suppose V'^{- , •) G C^''^{[0, T] x R") is a classical solution to the following Hamilton- 
Jacobi-Bellman equation: 



V,%t,x) + M U'>{t,x,u,V^{t,x),V°^{t,x)) = 0, {t,x) € [0,r] x R", 



(3.21) 



where 
Then 



V°{T,x) = h°{x), xeR", 

lH°{t,x,u,p,P) = {b{t,x,u),p)+tT [a{t,x,u)P] +g°{t,x,u). (3.22) 



V^it, x) < J°{t, x; u{-)), V(t, x) e [0, T] X R", u{-) S U[t, T]. (3.23) 
If {t, x) e [0, T) X R" is given and {X{-), «(•)) is a state-control pair with the initial pair {t, x) such that 

u{s) e avgmmU\s,X{s), - ,V°{s,X{s)),V°,{s,X{s))), s G [t,T], (3.24) 

then 

y"(t,x) = J°(f,.T; «,(■)) (3.25) 

and {X{-), u{-)) is an optimal pair of Problem (C) for the initial pair {t, x). 

We make some remarks on the above verification theorem. First of all, to guarantee (3.21) to have a 
classical solution V°{- , •), one may pose different conditions. A typical one is the uniform ellipticity condition: 

a{t, x, u) = ^a{t, x, u)a{t, x, uf > 6In, V(t, x, u) G [0, T] x R" x U, (3.26) 

for some 6 > 0. This condition implies that n < d and a{t, x, u) stays full rank for all {t, x, u). Thus, it does 
not include the case that (x, u) M- a{t, x, u) is linear, which is the case for LQ problems. On the other hand, 
for a standard LQ problem with deterministic coefficients, when the Riccati equation admits a solution P(-), 
the function V°{t,x) = {P{t)x,x) is a classical solution to the corresponding HJB equation for which the 
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uniform cllipticity condition fails. The similar situation happens for the classical Merton's portfolio problem. 
This observation shows that there are quite different conditions under which the corresponding HJB equation 
admits a classical solution. In the following sections, from time to time, we will simply say that the relevant 
HJB equation has a classical solution without getting into detailed sufficient conditions for that. Likewise, 
suppose there exists a map V° : [0, T] x R" x R" x .S" ^ such that 

Il°{s,x,i;°{s,x,p,P),p,P) = inf lf{s,x,u,p,P), \/{s,x,p,P) G [0,T] x R" x R" x 5". 
Then the pair {X{-),u{-)) appears in Proposition 3.6 is just a state-control pair satisfying the following 

' dX{s) = b{s,Xis),ij''{s,X{s),V,^{s,Xis)),V,l{s,Xis)))ds 

+a{sa{s),^"is,Xis),V°{s,X{s)),V°,{s,Xis)))sWis), s e [t,T], (3.27) 
_ X{t) = X, 

and we do not have to have the Lipschitz continuity of the map 

X ^ (b{s, X, ^0(5, X, x)XA-'^. x))),a{s, X, x, V°is, x), V,l{s, ,r)))) . 

In the following section, from time to time, we will just say some process is a solution to the relevant 
closed-loop system without mentioning if the drift and diffusion are Lipschitz continuous, etc. 



4 Time-Consistent Equilibria via Multi- Person Differential Games 

In this section, we are going to search time-consistent solution to Problem (N). Inspired by [27, 28], we will 
take an approach of multi-person differential games. 

To begin with, let us first introduce some necessary notions. Let ■p[0,r] be the set of all partitions 
n = {tfc I < < iV} of [0,T] with = to <ti <t2 < ■■■ < tjv-i <tN = T, and with the mesh size ||n|| 
defined by the following: 

||n|| = max {tk - tk-i). 

l<k<N 

We introduce the following important definition. 

Definition 4.1. A continuous map ^' : [0,T] x R" — >■ t/ is called a time- consistent equilibrium strategy 
of Problem (N) if for any {t, x) & [0, T) x R", 

I dX{s) = b{s,X{s),^{s,X{s)))ds + a{s,X{s),^{s,X{s)))dW{s), s € [t,T], ^^^^ 
I X{t) = X, 

admits a unique solution X{-) = X{- ; t, x, ^'(•)) such that for any t G [0, T), and £ > with t + s <T, 
j(i,X(i;0,a;,*(-));*(-)|[,,r]) 

< X(t;0,ar, *(•));«(•)«*(•) I [t+,,T]) +^(^)' VO G i + e], 
where R{e) represents a generic remainder term satisfying R{e) — > as e — > 0, and 



(4.2) 



with 



^ ^^l[t+e,T]JW I s€[t + s,T], 



dX^{s) =b{s,X'{s),u{s))ds + <j{s,X'{s),u{s))dW{s), s e [t,t + e), 

dX%s) =b{s,X%s),^{s,X'{s)))ds + a{s,X'{s),'i>{s,X'{s)))dW{s), s G [t + e,T], (4.4) 
[ X'{t)=X{t;0,x,^{-)). 
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In this case, we call X{-) = X{- ; 0, x, *!'(■)) a time- consistent equilibrium state process, call the corresponding 
■«(•) = \E'(-;X(-)) a time-consistent equilibrium control for the initial state x, and call {X{-),u{ )) a time- 
consistent equilibrium pair. Further, function V : [0,T]x R" ^ R is called the equilibrium value function of 
Problem (N) if 

V^(t,X(t;0,x, *(•))) =J(t,X(t;0,a;,*(-));*(-)|[t,T])' *e [0,r]. (4.5) 

Note that our definition of time-consistent equilibrium strategy is of closed-loop type. Thus, it is com- 
parable with that given in [5, 2, 3], and it is different from that in [13]. 

In what follows, the following definition which is equivalent to the above is more convenient to use. 

Definition 4.1'. A continuous map : [0,T] x R" U is called a time- consistent equilibrium strategy 
of Problem (N) if for any {t, x) G [0, T) x R", 



j dX{s) = b{s,X{s),^{s,X{s)))ds + a{s,X{s),^{s,X{s)))dW{s), s€ [t,T], 
I X{t) = X, 

admits a unique solution X{-) = X{- ; t, x, ^'(•)) such that for any H = {tk \ 1 < k < N} G P[0, T], 



(4.6) 



j(ife_i,X(ife_i;0,a;, *(■)); *(-)|[4,_^,T]) 

< j(ife_i,x(tfe_i;0,x,*(-));w'=(-)e*(-)|[,^,j,]) +i?(l|n||), (4-^) 

yu'^i-) GU[tk-i,tk], 

where R{r) represents a generic remainder term satisfying R{r) — as r — > 0, and 

^ \ ^(t,xHt)), tG[t„T], 

with 

dX''{s) = b{s,X''{s),u''{s))ds + a{s,X''{s),u''{s))dW{s), sG [tk-utk), 
dX>'{s) = b{s,X''{s),^{s,X>'{s)))ds + a{s,X''{s),^{s,X''{s)))dW{s), s G [tk,T], (4.9) 
_ X''{tk-i)=X{tk-i;0,x,^{-)). 

4.1 Multi-Person Differential Games. 

We now consider an A^-person differential game, called Problem (G^), as briefly described in Section 1. 
Throughout this section, we assume that (HI) (H3) hold. Let us start with Player N who controls the 
system on [tjv-itAr)- More precisely, for each {t,x) G [tjv-i,iAr] x R", consider the following controlled SDE: 

r dX^{s)^b{s,X^{sW{s))ds + a{s,X^{s),u^{s))dW{s), sG[t,t^], 
I X^it)=x, 

with cost functional 

J^(t, x; n^(-)) =^t[J^^ g{tN-i, s, X^(s), «^(s))rfs + /i(tjv-i, X^(ijv))] • (4.11) 

Note that 

J^(<^_i, x; zi^(-)) = J{tN-ux; zi^(-)), (x, w^(-)) S R" x U[tN-i,tN]. (4.12) 
We pose the following problem for Player N: 
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Problem (Cjv)- For any {t,x) € [tjv-i,tjv) x R", find a u^{-) = u'^{-;t,x) € U[t,tN] such that 



u" (■)eU[t,tN\ 



(4.13) 



The above defines the value function V^{- , •) on [fjv-i,tAr] x R", and in the case u^{-) exists, by (4.12), 
we have 

J{tN-i,x;u'^{-)) = V^{tN-i,x), Va;eR". (4.14) 
Under proper conditions, V^{- , •) is the classical solution to the following HJB equation: 



Vj^{t,x) + inf M{tN-i,t,x,u,V^{t,x),V^^{t,x)) = 0, {t,x) e [tiv-i,tjv] x R", 



(4.15) 



y"(ijv, x) = h{tN-i,x), X G R". 
By the definition of tp : D[0, T] x R" x R" x 5" -)■ C/ (see (3.16)-(3.17)), we may also write (4.15) as follows 

Vt^{t, x) + M{tN-i,t, X, ^{tN-i,t, X, V^{t, x), T/n (i, x)),V^{t, x), V^^{t, x)) = 0, 

{t,x)€[tN-i,tN]xU", (4.16) 
V^{tN,x) = h{tN-i,x), a;eR". 

Clearly, V^{- , •), well-defined on [tN-i,tN] x R", depends on In-i and fjv- With such a solution V^{- , ■) 
of (4.15) (or (4.16)), let us assume that the following closed-loop system admits a unique solution X^(-) = 
X^{- ; tN-i,x): (suppressing the dependence of X^{-) on tN through V^{- , •)) 



dX^(s) = b{s, X^(s), V(tiv-i, s, X^(s), V^{s, X^(s)), VZis, X^(s)))) ds 

+a{s, X^(s), i^itN-us, X^(s), V^{s, X^(s)), (s, X^(s))))dW(s), 

s G [fjv-i,tjv], 

L X^(tjv-i) = a;. 



(4.17) 



Then under (H3) and Proposition 3.6, an optimal control m^(-) of Problem (Cjv) for the initial pair {tN-i, x) 
admits the following feedback representation: 



n^(s) = «^(s; tM-i,x) = V(tiv-i, s, X^(s), V;n(s, X^(s)), (s, X^(s))) 

= V(iiv-i, s, X^'is; tN-i,x), V^{s, X^(s; tjv-i, x)), (s, X^(s; iiv_i, x))) (4.18) 

and -^^(O = X^{- ■,tN-i,x) is the corresponding optimal state process. 

Next, we consider an optimal control problem for Player {N — 1) on [tN-2,tN-i)- Naturally, for any 
initial pair (t,x) e [iAr_2, ^at-i] x R", the state equation should be 



dX^-\s)=b{s,X^-'{s),u^-\s))ds + a{s,X^-\s),u^-\s))dW{s), se[t,tN-i) 



X 



(t) = X. 



(4.19) 



To determine the suitable cost functional, we note that Player (A'^ — 1) can only control the system on 
[tjv-2, ^AT-i) and Player A'' will take over at t^-i to control the system thereafter. Moreover, Player {N — 1) 
knows that Player N will play optimally based on the initial pair (tjv-i, -X'^~^(fjv-i)) of Player A'', which 
is the terminal pair of Player (A'' — 1). Hence, the sophisticated cost functional of Player (A'' — 1) should be 



tN-I/ 



{t,x;u''-\-))=Et 



I. 



N-2, i>, 



S,X 



{s),u^-\s))ds 



+ / g{tN-2, s, X^(s; tjv-i, X^-'(tiv-i)), w^(s; tjv-i, ^"^"'(tiv-i))) ds 

+ h(tN-2,X^{tN',tis[-l,X^ ^(tjV-l))) 
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(4.20) 



Note that although Player (N — 1) knows that Player N will control the system on [tN-i,tN], he/she still 
"discounts" the future costs in his/her own way (see appearing in the running cost on [tN-i,tN] and 

in the terminal cost at tjv)- Now, if we denote 



^{x)='EtN-i / g{tN-2,s,X^{s;tN-i,x),u^{s;tN-i,x))ds + h{tN-2,X^{tN;tN-i,x)) 
then the cost functional (4.20) can be written as 



J^-\t,x;u^-'{-))=K 



tN- 



g{tN-2,s,X''-\s),u''-\s))ds + h''-\x''-\tN-i)) 



(4.21) 



We see that the optimal control problem associated with the state equation (4.19) and the cost functional 
(4.21) looks like a standard one. But, the map x h^~^{x) seems to be a little too implicit, which is difficult 
for us to pass to the limits later on. We now would like to make it more explicit in some sense. Inspired by 

the idea of Four Step Scheme introduced in [16, 17] for FBSDEs with deterministic coefficients, we proceed 
as follows. For the optimal state process {■) = X^ {■ ■,tN-i,x) determined by (4.17) on [tjv_i,fjv], we 
introduce the following BSDE: 



dF^(s) = -g{tN-2, s, X^(5), ^(iw_i, X^(s), V^{s, X^(s)), (s, X''{s))))ds 

+Z^{s)dW{s), se[ijv-i,tiv], 

[ Y''{tN)^h{tN-2,X''{tN)), 

which is equivalent to the following: 

f dY''{s) = -g{tN-2,s,X^{s),u''{s))ds + Z''{s)dW{s), s G [tN-iM, 

1 y^(i;v) = /l(ijV-2,^^(ijv)), 



(4.22) 



(4.23) 



Note that appears in the drift of BSDE and in the terminal condition. This BSDE admits a unique 
adapted solution {Y'^ {■), Z'^ {■)) = (y^(- ; a;), Z^(- ; a;)) ([17, 29]), uniquely depending on a; G R". Further, 
one has 



y^(ijv-i) = IEt„_, / g{tN-2,s,X''{s),u''is))ds + h{tN-2,X''itN)) =h''-\x) 



tN-1 



It is seen that (4.17) and (4.23) form an FBSDE. By [16] (see also [17, 29]), we have the following represen- 
tation for y^(-) 

Y^{s)^e^i.s,X^{s)), sG[tN-i,tN], 
as long as 6^(- , ■) is a classical solution to the following PDE: 



'ef (s, a;) + ( is, x), b{s, x, i^{tN-i,s, x, V^is, x), (s, a;))) ) 
+tr [a{s,x,il){tN-i,s,x,V^{s,x),V^^^{s,x)))e^^{s,x)] 

+g{tN-2,s,x,ip{tN-i,s,x,V^{s,x),V^{s,x))) =0, {s,x) e [ijv-i,ijv] x R" 
[Q'^{tN,x) = h{tN-2,x), xeR", 



or equivalently. 



ef (s, a;) + B.{tN-2, s, x, V'(tiv-i, s, x, V^{s, x), (s, a;)) , 6^ (s, x), O^Js, a;)) = 0, 

{S,X) e [tN-l,tN] X R", 

e^{tN,x) = h{tN-2,x), a;GR". 



(4.24) 



(4.25) 



(4.26) 
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Note that 0^(- , •) depends on (tjv-2,iAr-i,tjv)- We point out that in general, 



ptN 



7^ 



/ g{tN-2, s, U^{s))ds + h{tN-2, X^(T)) 

JtN-l 

f^'^ - - 1 

/ g{tN-i,s,X''{s),u''{s))ds + h{tN-uX''{T)) =V^itN-i,x). 

JtM-l 



(4.27) 



With the above representation 6 (• , •) of Y"{-), we can rewrite the cost functional (4.21) as follows: 



J^-i(t,x;«^-i(-)) = Et / g{tN-2.s,X^-\s),u''-\s))ds + Q^{tN-i,X''-\tN-i)) 



(4.28) 



Wc now pose the following problem: 

Problem (Cjv-i). For any {t,x) e [ijv-2, ^jv-i) x R", find a u^-^(-) = u^-'^{- ■,t,x) e i/[tjv-2, ijv-i] 
such that 

j''-\t,x;u''-\-))= _ inf j''-\t,x;u''-\-))^V''{t,x). (4.29) 

j"-i(-)6W[t,tiv-i] 



The above defines the value function V^{- , •) on [t7v_2, tjv-i) x II"- Under proper conditions, V^{- , ■) 
is the classical solution to the following HJB equation: 

Vt^{t,x) + inf M{tN-2,t,x,u,V^{t,x),V^^{t,x)) = 0, {t,x) G [tjv-2,ijv-i) x R", 

(4.30) 

y"(ijv_i - 0, a;) = e^(ijv_i, a;), a; e R". 
By the definition of the map ipi') again (see (3.16)-(3.17)), we may also write the above as 

Vt^{t, x) + H{tN-2, t, x,^p{tN-2, t, X, V^{t, x), (t, x)) , V^{t, x),V^,{t, x))=Q, 

{t,x)e[tN-2,tN-i)xR", (4.31) 

V^{tN-l-0,x) = e'^{tN-l,x), XGB". 

From (4.27), we see that in general, 

V^{tN-i -0,x) = Q^{tN-l,x) V^{tN-UX). 

Thus, V^{- , •), which is now defined on ^at] x R", may have a jump at {tjv-i} x R". For any x € R", 

suppose the following admits a unique solution X^~^{-): 

'rfX^-i(s) = b[s, X^-i(s), V(tjv-2, s, X^-i(s), V^{s, X^-i(s)), (s, X^-i(s))))ds 

+c7(5,X^-\s),V(iiv-2,s,X^-\s),yn(s,X^-V)),Fn(s,X^-i(s))))rfiy(s), ^432^ 

S G [iAr-2,*JV-l)j 

X''-\tN-2) = X. 

Then we define 



«^-i(s; tN-2, x) = i^{tN^2,s, X^-i(s), V^{s, X^-i(s)), (s, X^-i(s))) , 

= V(ijv-2, s, X^-i(s; tw-2, x),V^is, X^-\s; 1^-2. x)), V;^ (s, X'^-^s- tjv-2, x))) , 

S G \tN-2, tN-l), 

which, again by Proposition 3.6, is an optimal control of Problem (Cat-i) with the initial pair (tN-2,x). 
Now, for the optimal pair 

(X^-1(.),«^-^(-)) = {x''-\-;tM-2,x),u''-\-;t^-2,x)) 
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of Problem (Cjv-i) (on [tjv-2, tjv-i]), we make a natural extension on [tjv-i,iAr] as follows: 

1 «^-i(s)=«%;tjv-i,X^-^(tjv-i)), 
Thus, the extended satisfies 

S G [fjv-l,tjv], 

(^X^-i(i;v-2) = a;. 

We refer to such a pair (^X^~^{-),u^~^{-)) as a sophisticated equilibrium pair on [tjv-2,iAr]- Let 

fe=i 

It is easy to see that 

< s-^n(s) < ||n||, se[0,T]. (4.36) 
Then (4.34) can be written compactly as 

'dX^-^(s) = X^-i(s), V'(i?n(s), X^-i(s), V;n(s, X^-i(,sO), (s, ^ 

+a(s,X^-i(s),V'(^"(s),5,X^-i(s),Fn(s,X^-H.)),Fnjs,X^-^(s))))dW^(s), (4.37) 

S e [tN-2, ijv], 

Also, one has 

J{tN-2,x;u''-\-))=-Et^_, / ff(t;v-2,s,X^-i(s),u^-i(s))ds 

JtN-1 

- - n f4 38) 

+ / g{tM-2,s,X^-\s),u''-\s))ds + h{tN-2,X''-\T))\ 

= J^-i(tjv-2,a;;w^-i(-)) = V^{tN-2,x), x G R". 
We point out that in general it may happen that, 

j{tN-2,x-u^-\-)) > mi j{tN-2,x;u{-)), (4.39) 

u(-)eW[tiv-2,tAr] 

which means that the the sophisticated equilibrium pair might not be an optimal pair (for the given initial 

pair). 

Similar to the above, in order to state an optimal control problem for Player (iV — 2) on [tN-3, tN-2], we 
introduce the following BSDE on [tN-2,tN]- 

idY^-\s) = -g{tN-3,S,X''-\s),u''-\s))ds + Z''-\s)dW{s), SG[tN-2,tN], 
\Y''-\tN) = h{tN-3,X''-\tN)), 
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where (^X^ ^i')) ^^"^ sophisticated equihbrium pair determined by (4.37) on [tN-2,tN], uniquely 

depending on a; € R". Let {Y^-^{-), Z^-\-)) = {Y^-^{- ■,x), Z^-\- -x)) be the adapted solution of this 
BSDE. Then (4.37) and (4.40) form an FBSDE. Similar to the above, we have 

Y^-'is) = e^-i(,s,X^-i(s)), s G [tN-2,tN], (4.41) 

as long as 0^^^(- , •) is the solution to the following PDE: 

'Q^-\s,x)+M{tN-3,s,x,^{e''is),s,x,V^is,x),V^,{s,x)),e^-\s,x),Q^-\s,x))=0, 

{.s,x) e[tN-2,tN]xWC', (4.42) 

e^-\tN,x) = h{tN-3,x), XGR". 

Having the above preparation, we now consider, for any {t,x) G [tjv-Si iAr-2) x R") the state equation 

f dX^-^{s) = b{s,X''-^{s),u''-\s))ds + a{s,X''-^{s),u''-^{s))dW{s), sG[t,tN-2], 
1 X^-^{t)=x, 

and the (sophisticated) cost functional 



(4.43) 



r-'{t,x;u''-'i-))=Et[ j gitN-3,s,X''-'{s),u''-'{s))ds + e"-\tN-2,X''-'{tN-2))\. (4.44) 

We pose the following problem for Player {N — 2): 

Problem (Cjv-2). For any {t,x) G [tN-3,tN-2) x R", find a u^-'^{-) = u^-'^{- ■,t,x) G W[tjv-3, 
such that 



J^-2(f, x; w^-'(-)) = inf J'^'^it, x; w^-'(-)) = V^it, x). 

«"-=^(-)eW[tiV-3,fiV-2] 



(4.45) 



Together with the previous definition, we see that V^{- \s now well-defined on [fjv-3,iiv] x R". Under 
proper conditions, V^{- , ■) is a classical solution to the following HJB equation: 

V^{t,x)+ ini M{l^{t),t,X,U,V^{t,x),V^,{t,x)) =0, {t,x) G [i;v-3,tiV-2) x R", 

ueu (4.46) 
V^{tN-2-Q,x)=Q^-^{tN-2,x), a;GR". 

Further, by the definition of the map V(')' ^^^o write the above as 

V^{t, x) + H(^n(i), x,i,{F{t),t, X, V^{t, x), (i, x)) , V^{t, x), (t, x)) = 0, 

(t,a;) G [i;v-3,tiv-2) xR", (4.47) 

_ V^{tN-2-0,x) = e''-\tN-2,x), XGR". 

Also, similar to (4.27), we have that in general, 

V^{tN-2 - 0, ar) ^ V^{tN-2,x). (4.48) 

The above procedure can be continued recursively. By induction, we can construct sophisticated cost 
functional J^(t, a;; u^(-)) for Player k, and 

V^{t,x) = inf J'=(i, a;; «'=(•)), {t,x) G [tk-utk) x R", 1 < k < N, (4.49) 

M'=(-)ew[t,tfc] 

with the value function V^{- , •) satisfying the following HJB equations on the time intervals associated with 
the partition H: 

V.^'it, x) + Mii^'itU X, V'(^"(t), t, x, V^{t, x), V^^it, x)) , V^{t, x), (t, x)) = 0, 

(t,a;) G [iiv-i,tiv] xR", (4.50) 

_ V^{tN,x) = h{tN-i,x), xgR", 
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and for k = 1,2, ■ ■ ■ , N - 1, 

V^{t, x) + M{f'{t),t, x,i^{f'{t), t, X, V^{t, x), (t, x)) , V^{t, x), (t, x)) = 0, 

{t,x)€[tk-utk)xlC', (4.51) 

_ V''{tk-0,x) = e''+\tk,x), a;eR", 
where, for fc = 1, 2, • • ■ , A'' — 1, 6*'+^(- , •) is the solution to the following (linear) PDE: 

e^i(t, x) + Ii{tk-i,t, a;, V(^"(t), t, X, V^it, x), T/" (t, x)) , Ql+\t, x), e^+i(f, x)) = 0, 

(f,x)€[tfc,tjv]xIR", (4.52) 
id^+\tN,x)=h{tk-i,x), a;€R". 

Now, we define 

^^{t,x) ^ i,{i''{t),t,x,V^{t,x),V^At,x)), {t,x) G [0,r] xR". (4.53) 
Then for any given x S IR", let X^{-) be the solution to the following closed-loop system: 



dX^{s) = b{s, X^{s), ^^{s, X^{s)))ds + a{s, X^{s), ^"(s, X^{s)))dW{s), s e [0, T], 
^"(0) = X, 



(4.54) 



and denote 

vP(s) = ^^{s,X^{s)), s e [0,r]. (4.55) 
According to our construction, we have 

J(t.-i,^"(ife-i);*"(-)|[,,_,,T]) = J(tfc-i,Xn(f,_i);«n(.)|^^^_^^^^^) =yn^^^_^^^n(^^_^)) 
= J^(t._„xn(i,_i);tz"(.)|[,^_^,J 



inf j''{tk-i,X^{tk-i);u>'{-)) < J^(tfe_i,Xn(tfc_i); «*=(•)) 

u''(-)euitk-i.tk] 

j(ffe_i,xn(ifc_i);u'=(-)©*"(-)|[t„T])' \fu\-)eU[tk-i,tk], l<k<N, 



(4.56) 



where u''{-) © *"(•)! j,^ is defined the same way as (4.8)-(4.9). Similar to (4.39), for A; = 1, 2, • • • , A'' - 1, 
we have in general that 

j{tk-i,X''{tk-i);u''{-))> ^inf J{tk-i,X''{tk-i);u{-)). (4.57) 

Since the involved N players in Problem (G^) interact through the initial/terminal pairs {tk,X{tk)), k = 
1,2, - ■ ■ ,N — 1, one should actually denote 

j'=(tfc_i,X(ffe_i);«'=(.)) =P(x;ui(-),---,«'^(-)), l<fc<iV. (4.58) 
Hence, (4.56) means that if we let 

^'(•)=«"(-)|[.,_,,,p l<k<N, 

then {v}{-), ■ ■ ■ ,u^{-)) is a Nash equilibrium of the A^-person non-cooperative differential game associated 
with j''{x;u^{-),---,u'^{-)), l<k<N (defined in (4.58)). 
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4.2 The formal limits. 

We now would like to look at the situation when ||n|| — >■ 0. Suppose we have the following: 

lim (\V''{t,x)-V{t,x)\ + \V^{t,x)^V^{t,x)\ + \V^,it,x)-VUt,x)\) =0, (4.59) 
||n||-»-o V / 

uniformly for {t,x) in any compact sets, for some V{- , ■). Under (H3), we also have 

lim \^^(t,x) - *(t,a;)| = 0, (4.60) 

||n||-)-o 

uniformly for {t, x) in any compact sets, for 

^{t,x) = 7p{t,t,x,V^{t,x),V^^{t,x)), {t,x) e[0,T]xWC. (4.61) 
Then the following limit exist: 

l|nmJ|Xn(.)-X(.)|U.(a;C([0,T];Rn))=0, 



for X{-) solving the following SDE: 



dX{s) = b{s,X{s),u{s))ds + a{s,X{s),u{s))dW{s), sG [0,T]. 
X{0) = X, 



Clearly, 

By (4.56), we have 



j{e^{t),x^{e^mu^{-)) = yn(^n(f),xn(t)), t G [0,T]. 

Thus, passing to the limits, we have (4.5). Also, we have the following: 

J(tfc_i,X(ifc_i);*(-)|[,^_^_^]) = J(tfc-i,X(tfc-i);n(-)|[,^_^,^]) 



E 



tk-i 



tk-i 



T 

g{tk-i,s,X{s),u{s))ds + h{tk-i,X{T)) 

tk-i 

T 



+Rm\ 



(4.62) 



where 

uis) = ^{s,X{s)), sG[0,r], (4.63) 

and 

Ll{n,C{[0,T];lC')) = |x : [0,r] X ^ R" I X{-) has continuous paths, 



E[ sup |X(i)|2] < 
te[o,T] 



hm =0. (4.64) 

l|n||-*-o 



/ g{tk-i,s, X^{s), u^{s))ds + h{tk-i,X^{T)) 

=v''{tk-,,x^{tk-i))+Rm\)=j'{tk-i,x^{tk-i)-,u^^^^^ 

< j''{tk-i,X^{tk-i),u\-)) + R{m\) 

= j(tfc_i,x(tfc_i);M'=(-)e*(.)|[,^ j,])+i?(||n||), ^u\-) eu[tk-utkl 

for some R{r) with R(r) — >■ as r — >■ 0. Hence, by Definition 4.1, *(• , •) is a time-consistent equilibrium 
strategy, and V{- , •) is a time-consistent equilibrium value function of Problem (N). 
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In the rest of this subsection, we will formally pass to the limits to find the equations that can be used to 
characterize the equilibrium value function V{- , •). To this end, let us first write the equations for 6'^"'"^(- , •) 
in the integral forms: For fc = l,2, ••■,A'' — 1, one has 

Q''+\t, x) = h{tk-i,x)+£M{tk-r,s, x,^{i^{s),s, x,V^{s, x),V^,{s, x)) ,Q''+\s, x),Q''+\s, x)) ds 

= h{h_„x) +f{{ e'+'G'^, 2;), b{s, X, s, X, V^{s, x), (s, x))) ) 

+tr [a{s, X, ^(^"(s), s, X, V^{s, x), (s, x))) O^f (s, x)] 
+g{tk-i,s, X, ■ip{£^{s),s, X, V^{s, x), V;" (s, x))))ds, 

{t,x) e [tk,T]xir. 



Let us define 



en(r,t,x) = 2;)J[i,_„t,)(r), {T,t,x) e D[0,T] x R". 



fe=i 



Then 



@^{T,t,x) = h^{T,x) 



{{e^{T,s,x),b{s,x,^P{e''{s),s,x,V^{s,x),V.^,{s,x)))) 

+tr [a{s, X, s. x. V^{s, x), VjUs, x)))e^,{T, s, x)] 

+g''{T, s, X, s, X, V^{s, x), yn (s, x))) ]ds, 



where 



N-l 



h^{T,x) = J2 Htk-ux)I[t,_,,t,){T), {r,x) e [0,T] X R", 



fe=i 



JV-l 



g^{T,s,x,u) = ^ g{tk-i,s,x,u)I[t^_^^tk){T), {t,s,x,u) G D[0,T] x R" x JJ. 



fc=i 



(4.65) 



(4.66) 



(4.67) 



Let us look at V^{- , •). For {t,x) e [tN-i,tN] x R", we have 

V''{t,x) = h{tN-ux)+J^M{tN-us,x,ij{e''{s),s,x,V;^{s,x),V;^,{s,x)),V^{s,x),V^^^^^ 

= h{£^{t), x)+jy{£^{t), s, X, ^{i'^is), s, X, V^{s, x), (s, x)), V^{s, x), (s, x))ds, 
and for {t,x) e [tk-i,tk) x R", fc = 1, 2, • • • , iV - 1, 

V^{t, x) = e'^+i {tk,x) + j^^M (^"(s), s, X, V(^"(s), s, X, Vf(s, x), V^,{s, x)) , V^{s, x), (s, x)) ds 

= Q^{tk-utk,x) + J''{{ x), b{s, X, i^{F{s), s, x, V^is, x), (s, x))) ) 

+tr [a{s, X, i^ii^is), s, x, V^is, x), V^,{s, x)))V^,is, x)] 
+g''{tk^i,s, X, i^ii^'is), s, X, V^is, x), 1/n (s, x))) }ds. 

On the other hand, for {t,x) S [tk-i,tk) x R", we have 

e^'itk-iA x) = e^{tk-i,tk, x) + j" [{ @^{tk-i,s, x), b{s, X, V'(^"(s), s, x, V^is, x), (s, x))) ) 

+tr [a{s, X, ^Pii'^is), s, x, V^{s, x), (s, x)))eZ{tk-i,s, x)] 
+g^ [tk-i, s, X, V(^"(s), s, x, l/n(s, x), (s, x))) }ds. 
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(4.68) 



Therefore, in the case that (4.67) is well-posed, we have 

V^{t,x) = e^{tk-i,t,x), {t,x)e[tk-i,tk)xWr, (4.69) 

or equivalently, 

V^{t, x) = e^{i^{t), t, x), {t, x) e [0, ijv-i) X R". (4.70) 
Now, let us assume (4.59) holds for some V{- , ■) and 

lim (\Q^{T,t,x) - eiT,t,x)\ + \e4T,t,x) - Q4T,t,x)\ + \e^4T,t,x) - e^4T,t,x)\) = o, 

l|n|Ko \ / (4.71) 

{T,t,x) e D[0,T] X R", 

for some ©(•,•,•). Let us assume that 

\hr{T,x)\ + \gr{T,t,x,u)\ < K, V(r, cc, w) S D[0, T] X R" X U". (4.72) 

Then we have 



lim h^{T, x) = h{T, x), (r, x) G [0, T] x R", 
||n||-)-o 



and 



(r,s,a;) G £'[0,T] x R". 
Consequently, we obtain the following integro-partial differential equation for &{■,■,■): 

e{T,t,x) = h{T,x) + j M{T,s,x,'il){s,s,x,@x{s,s,x),e^^{s,s,x)),@^{T,s,x),Q^^{T,s,x))ds, ^^^^^ 

{T,t,x) G D[0,T] X R", 

and relation 

V{t, x) = e(i, t, x), {t, x) G [0, T] X R". (4.74) 
It is clear that (4.73) is an integral form of the following differential equation: 

Qt{T, t, x) + ]H(r, t, X, tjj[t, t, X, Qa:{t, t, x), Qxx{t, t, x)) , Qx{t, t, x), <9xx{t, t, x)) = 0, 

{T,t,x) € D[0,T]x'R", (4.75) 

e(r, T, x) = h{T, x), (r, x) G [0, T] x R", 



where 

{]H(t, t, x,u,p,P) = tr [a{t, x, u)P] + { b{t, x,u),p) +g{T, t, x, u), 
ip{T,t, x,p, P) G argmin]H(T, t, x,- ,p,P). 
Therefore, we may equivalently write (4.75) as follows: 

et(r, t,x) + { e,(T, t, x),b{t, x, i^{t, t, X, e^it, t, x), e,,(t, x))) ) 

+tr [a{t, X, Tp{t, t, X, Qxit, t, x), Qxxit, t, x)))Qxx{t, t, x)] 

+g{T,t,x,i,{t,t,x,Qx{t,t,x),Qxx{t,t,x))) =0, {T,t,x) G D[Q,T] x R", 

L Q{t,T,x) = h{T,x), (T,a;) G [0,T] x R". 



(4.76) 



(4.77) 



We call the above (4.77) the equilibrium Hamilton-Jacobi-Bellamn equation (equilibrium HJB equation, for 
short) of Problem (N). If one can find 0(- , ■ , ■) from the above, then the equilibrium value function V{- , ■) 
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can be determined by (4.74), and the (time-consistent) equilibrium pair can be determined by (4.62) and 

(4.63), in principle. 

Let us make some remarks on (4.77). 

(i) It is an interesting feature of (4.77) that both 0(r, t, x) and Q{t, t, x) appear in the equation where the 
later is the restriction of the former on r = t. On one hand, although the equation is fully nonlinear, due to 
the fact that Q{t,t,x) is different from Q{T,t,x), the existing theory for fully nonlinear parabolic equations 
cannot apply directly. On the other hand, it is seen that if Q{t,t,x) is obtained from an independent way, 
then (4.77) is actually a linear equation for 0(t, t,x) with t can be purely regarded as a parameter. 

(ii) In the case that V{tp) is not equal to ^[O, T] x R" x R" x 5", the condition 

(r, t, X, {t, t, x) , {t, t,x))G V{iP) (4.78) 

has to be regarded as a part of the solution. We will see that for some interesting special cases, the above 
condition can come automatically. More generally, we may also write (4.75) as 

9f (r, t, x) + ]II(t, t, X, argminH(t, i, x, ^^{i, t, x), Qxx{t, t, x)) , &x{t, t, x),Qxx{t, t, x)) 9 0, 

(r,t,a;) gD[0,T] xR", (4.79) 
e(T, T, a;) = /i(t, x), (t, a;) € [0, T] x R", 

since the set argmin]H(t, t, x, Qx(t, t, x), Oxx(t, t, .x)) might contain more than one point. 

(iii) In the case ^{tp) ^ V(H), according to Lemma 3.2, we can define 

]H^(r,i,a;,u,p,P) = ]H(r,i,a;,u,p,P) 
and there exists a : £>[0, T] x R" x R" x <S" Z7 such that 

(r, i, X, V" (r, t,x,p,P), p, P) = inf (r, t , x, u, p, P) = H^T,t,x, p, P) . 

u£(7 

Further, 

lim H''{T,t,x,p,P) = H{T,t,x,p,P), limelip" {T,t,x,p,P)\'^ = 0. 

e— >0 e— 

It is not hard to see that the above actually amounts to defining 

g^{T, t, X, u) = g{T, t, X, u) + e|u|^. 

We may refer to the corresponding problem as a regularized problem. If the corresponding equilibrium value 
function is denoted by V^{- ,■), then it is expected that 

lim V^t, x) = V(t, x), {t, x) 6 [0, T] X R". (4.80) 
£4-0 

However, in general, if (X^(-), w'^(-)) is an equilibrium pair for the regularized problem, we might not have the 
limit of u^{-) as e — )• 0. In this case, we should be satisfied by the above characterization of the equilibrium 
value function V{- ,■), and m^(-) can be regarded as some kind of "near equilibrium control". 

(iv) For the case 

a{t,x,u)=a{t,x), V(t, X, u) e [0, T] X R" X U", (4.81) 
i.e., the control does not enter the diffusion of the state equation, tjj{-) is independent of P and 

'4'{t,x,p) e a.rginm[{p,b{t,x,-))+g{T,t,x,-)]. (4.82) 
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Then the equilibrium HJB equation can be written as 



@t{T,t,x)+tv [a{t,x)Qxx{T,t,x)] + {b{t,x,il){t,t,x,Q^{t,t,x))),Q^{T,t,x)) 

+g{T,t,x,'il;{t,t,x,Q^{t,t,x))) =0, (r,t,a;) G £>[0,T] x R", 

e(T, T, x) = h{T, x), (r, x) G [0, T] X R". 



(4.83) 



We will carefully discuss this case in the next section. Note that for a deterministic problem, namely Problem 
(N) for an ordinary differential equation system, we may take 

a{t, X, u) = si, {t, X, u) G [0, T] X R" X U, 

for e > to regularize the problem. Then the corresponding equilibrium HJB equation reads 



(r, t, x) + - Ae|^(T, t,x) + { b{t, X, ^{t, t, X, ei{t, t, x))) , Q%{t, t, x) ) 



+g{T, t, X, il){t, t, X, e|(t, t, x))) = 0, 
_ e^(r, T, x) = h{T, x), (r, x) G [0, T] x R". 



{T,t,x) G D[0,T] X R", 



It is expected that 0^(- , • , •) — )• 6(- , ■ , ■) in some sense, as e — >■ 0, with 

et(r, t,x) + { b{t, X, tp{t, t, X, e^it, t, x))) , e^ir, t, x) ) +g{T, t, x, i^{t, t, x, Q:o{t, t, x))) = 0, 

(r,i,x) gD[0,T] X R", 

e{T,T,x) = h{T,x), (t,x) G [0,r] X R". 
At the moment, it is not clear to us how one can define viscosity solution to the above equation. 



(4.84) 



(4.85) 



5 Well-Posedness of the Equilibrium HJB Equation 

In this section, we discuss the well-posedness for the equilibrium HJB equation (4.77). Let us first intuitively 
describe our idea. For any smooth function w(- , •), denote 



[C{t, v{t, ■))(p{-)] {x) = tr [a{t, X, ip{t, t, x, Vx{t, x), v^^{t, x))ip^x(x)\ 

+ {b{t,x,i){t,t, {t,x))),(px{x)), (t, a;) G [0, T] X R" 



and 



g{T,t,v{t,-)Kx) ^ g{T,t,x,i;{t,t,x,v^{t,x))), {T,t,x) € D[0,T] x FT . 
Consider the following linear abstract backward evolution equation: 



(5.1) 



(5.2) 



(5.3) 



1 et{T,t) + c{t,v{t))e{T,t) + g{T,t,v{t)) = o, te[T,T], 

1 e(T,T)=/i(r). 

Under some mild conditions, the above is well-posed, and we have the following variation of constant formula: 



Q{T,t)=£{T,t;v{-))h{T)+ £{s,t;v{-))g{T,s,v{s))ds, t e [t,T], 



(5.4) 



where £(•,•; «(•)) is called the backward evolution operator generated by £(• , «(•)). Consequently, the (time- 
consistent) equilibrium value function V{t,-) = Q{t,t,-) should be the solution to the following nonlinear 
functional integral equation: 

V{t)=S{T,t;V{-))h{t) + j £{s,t;V{-))g{t,s,V{s))ds, iG[0,T]. (5.5) 

25 



We call (5.5) the equilibrium HJB integral equaMon for Problem (N). Once a solution V{- , •) of (5.5) is found, 
we can, in principle, construct a (time-consistent) equilibrium control and an equilibrium pair for Problem 
(N). Of course, if we like, we may also solve the equilibrium HJB equation (4.77), which actually is not 
necessary as far as the construction of a time-consistent equilibrium pair is concerned. 

The well-posedness of (5.5) seems to be difficult for the general case. At the moment, we do not have a 
complete solution for that and hopefully, we can present some satisfactory results for the eqiiilibrium HJB 
integral equation (5.5) in our future publications. On the other hand, in the rest of this section, we are going 
to present a well-posedness result for an interesting special case of (5.5), from which one can get some taste 
of the problem. The main hypothesis that we will assume below is the following: 

a{t,x,u) = a{t,x), {t,x,u) €[0,T]xW xU, (5.6) 

namely, the control does not enter the diffusion of the state equation. As we discussed in Section 4, in this 
case, our equilibrium HJB equation reads 

@t{T, t, X) + ^tr [a{t, X)a{t, x)'^@;cx{t, t, x)] + { b{t, X, 'tp{t, t, X, Q^^it, t, x))) , @a:{T, t, x) ) 

+g{T,t,x,i;it,t,x,QS,t,x))) =0, {T,t,x) e D[0,T] xWC, ^^''^^ 

e(T, T, x) = h{T, x), (r, x) e [0, T] X R". 

The essential feature of (5.7) is that Qxx{t,t,x) does not appear in the equation (although @xit,t,x) still 
appears there). This leads to the well-posedness problem much more accessible. Further, from Example 3.5, 
we see that there are cases for which is as smooth as the coefficients and b(t,x,ip{t,t,x,p)) is boimded. 
Therefore, the case that we are going to consider below, although very special, includes a big class of 
problems. 

To avoid heavy notations, let us consider the following equation 

Qt{T,t,x) + tr [a{t,x)Qxx{T,t,x)] + {b{t,x, Qx{t,t,x)) ,Qx{T,t,x) ) +g{T,t,x,&x{t,t,x)) = 0, 

{T,t,x)€D[0,T]xlC', (5.8) 
e(T, T, x) = h{T, x), (r, x) £ [0, T] x R", 

with 

a{t,x) = ^a{t,x)(T{t,xf , {t,x) e [0,T] x R", 

b{t,x,p) = b{t,x,ilj{t,t,x,p)), (i,a;,p) e [0,T] X R" X R", 
g{T,t,x,p) = g{T,t,x,ip{t,t,x,p)), iT,t,x,p) G D[0,T] x R" x R"". 

To investigate the well-posedness of (5.8) above, let us make some preparations. Let C"(R") be the space 
of all continuous functions (p : R" — > R such that 

\\<f\\a = Mo + Ma < OO, 

where 

\\(p\\o = sup \<p{x)\, [<p\a= sup 1- . 

xeR" XjyefV^jXjty \x — y\ 

Further, let Ci+"(R") and C2+"(R") be the spaces of aU functions : R" -)■ R such that 

ll'y^=lll+a = y\\o + WfxWo + [ifixja < OO, 

and 

||</'||2+a = ||<P||0 + W'PxWo + W'PxxWo + i'Pxxja < OO, 

26 



respectively. Next, let B([0, T]; C"(]R")) be the set of all measurable functions / : [0,T] x R" -)■ R such 
that for each t G [0,T], f{t, •) G C"(R") and 

||/(-,-)||s([0,T];C»(R")) = sup •) ||a < OO. 

*e[o,T] 

Also, we let C([0, T]; C"(R")) be the set of all continuous functions that are also in B{[0, T]; C"(R")). Thus, 

C([0,T]; C"(R")) C B{[0, T]; C"(R")). 

Similarly, we define B([0, T]; C*=+«(R")) and C([0,T]; C"=+"(R")), respectively, for fc = 1,2. 
We introduce the following hypotheses for the above equation (5.8). 

(P) The maps a : [0,T] x R" ^ 5", 6 : [0,T] x R" x R" R", 5 : D[0,T] x R" x R" ^> R and 
h : [0, T] X R" R are continuous and bounded. Moreover, there exists a constant L > such that 

\ax{t,x)\ + \ba;{t,x,p)\ + \gx{T,t,x,p)\ + \bp{t,x,p)\ + \gp{T,t,x,p)\ + |/ix('r,a;)| < L, 

(r,i,x,p) G Z)[0,T] X R" X R". 

Further, a{t,x)^^ exists for all {t,x) G [0,T] x R" and there exist constants Aq, Ai > such that 

Xol < a{t, x)-^ < Xil, V{t, x) G [0, T] x R". (5.11) 

We point out here that some of the conditions assumed in (P) can be substantially relaxed. However, 
we prefer not to get into those generalities for the sake of simplicity in our presentation. Note also that 
typically, the ellipticity condition of a{t, x) looks like 

a{t, x) > SI, V(i, x) G [0, T] x R", 

for some 5 > 0. It is clear that when a{- , •) is assumed to be bounded, then the above is equivalent to (5.11). 
The number Aq in (5.11) will be used below. 

For any v{- , •) G C([0, T]; Ci+"(R")), we consider the following linear PDF, parameterized by r G [0, T): 

I Qtir, t, x) + C[t, v{-)]Q{t, t, x) +g{T, t, x, v^it, x)) = 0, {t, x) G [0, T) x R", 
\ Q{t,T,x) = h{T,x), a; G R", 

where the differential operator C[t,v{-)] is defined by the following: 

C[t,v{-)]ip{x) = ti [a{t,x)ipxx{x)] +{b{t,x,Vx{t,x)),ipx{x)), V(^(-) G C2(R"). (5.13) 

In what follows, we let 

Ca(R") = {(fii-) G C(R") I sup e-^l"l'|^(:r)| < ooj. (5.14) 

We have the following result whose proof follows a relevant one found in [9], with some minor modifications. 

Proposition 5.1. Let (P) hold and fix a r G [0, T). Tiien for any v{- , •) G C([0, T]; C^+"(R")) and any 
h{T,-) G Ca(R") with A < Cauchy problem (5.12) admits a unique classical solution 6(t, ■,■) and the 
following representation holds: 

(5.15) 



e{T,t,x)= I r<\t,x;T,y)h{T,y)dy+ f I r''^-\t,x;s,y)g{T,s,y,Vx{s,y))dyds, 



R" Jt JR 



{t, x) G [0, T] X R" 
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Here, T'"^'\t,x;s,y) is defined on [0,T] x R" x [0,T] x R" with t < s having the following properties: 

(i) For any fixed (s, y) e (0, T] x R", 

ri^-\t,x;s,y), r<\t,x;s,y), r,^-\t,x;s,y) 
are continuous in {t, x), (s, y) G [0, T] x R" with t < s, and for any fixed {s, y) e (0, T] x R", 

C[t,v{-)]T<\t,x;s,y) =0, (t, cc) G [0, s) x R", (5.16) 

(ii) For any ip{-) e Ca(R") with A < ^, 

lim / r-^-) {t, x; s, yMy)dy = <fi{x), a; G R", (5.17) 

The map T^^'^ {t, x; s, y) in the above proposition is called the fundamental solution to the problem (5.12). 
Our main result of this section is the following. 

Theorem 5.2. Let (P) hold. Then (5.8) admits a unique solution Q{- ,-,•)• 
Proof. Let 

Co{t)ip{x) = tr [a{t,x)ip^4x)], G C2(R"), (5.18) 

which is independent of v(-). Then 

C[t, vi-)]^ix) = Co{tMs) + { Kt, X, v^t, x)), Mx) ), V^(-) G C2(R"), (5.19) 

and (5.12) can be written as (fix r G [0, T)) 

@t{T, t, x) + Co{t)Q{T, t,x) + { b{t, X, Vx{t, x)), ©x(t, t, x) ) +g{T, t, X, Vx{t, x)) = 0, 

(t,a;)G [0,T)xR", (5.20) 
e{T,T,x) = h{T,x), X G R", 

Applying Proposition 5.1, we have 

e(T, t, x) = j^^ r°{t, x; T, y)h{T, y)dy + j^^ ^0(4, x; s,y)[{ b{s, y, w^(s, y)), e^(r, s, y) ) ^g^i) 

+fif(r, s, y, Vx{s, y))] dyds, {t, x) G [r, T] x R", 
where r°(t, x; s, y) is the fundamental solution of £o(')> given by the following explicitly: 

T°{t,x;s,y) = -e ^ , 

{Anis-t))-{det[a{s,y)]y (5.22) 

{t,x),{s,y) G [0,T] X R", t < s. 

Direct computations show that ([9]) 

\T\t,x;s,y)\ < _ e ^(-*) , 

\rl{t,x;s,y)\ <- -^e , 

(s-t) — 



Moreover, 



r°(i,a;;s,y) = -r^(i,x;s,y) - r"{t,x;s,y) (det [a(s,2/)])j^ + ( [a{s,y) ^]y{x - y),x - y) 
= -r° (t, x; s, y) - r°(t, x; s, y)p{t, x, s, y), 

28 



(5.24) 



where 



( [a{s,y) ^]y{x-y),x-y) 



/ {Hs,y) \Ax-y),x-y)\ 
{[a{s,y)-\^{x - y),x - y) 



and 



p{t, x,s,y) = -{ det [a(s, y)] ) ^ + 



\n{x-y),x-y) J 
( [a{s,y)~'^]y{x - y),x - y) 



A{s-t) 



Under (P), we see that 

\p{s, x,y)\<K(l + , V(s, X, y) G [0, T] x R" x R". 



(5.25) 



Then 



e^(T, t,x)= [ rl{t, x; T, y)h{T, y)dy +11 r° (i, x; s,y)[{ b{s, y, v^{s, y)), e^(T, s, y) ) 

+9{t, s,y,Vx{s,y))]dyds 



= - [ r°{t, x; T, y)h{T, y)dy - [ r^{t, x; T, y)p{t, x, T, y)h{T, y)dy 

+ / / Tl{t,x;s,y)[{h{s , y, Vx{s, y)),&x{T, s, y) ) +g{T, s, y, v^is, y))] dyds 

Jt JR" 

= [ r^{t,x;T,y)hy{T,y)dy- [ T°{t,x;s,y)p{t,x,T,y)h{T,y)dy 
Jr" Jr" 



+ 



t JR" 



r° (t, x; s,y)[{ b{s, y, Vx{s, y)),Qx{T, s, y) ) +g{T, s, y, Vx{s, y))] dyds. 



Hence, 



\QAr,t,x)\ < 



Ke 4(T-t) 
R" (T-t)^ 



\hy{r,y)\ + {l 



\x — y\' 
T ~t 



)\h{T,y)\ 



dy 



+ 



It Ir" 



Ke 



(|0x(t, s,y)\ + l)dyds 



< 



Ir' 



Ke 



(l+^-^f^)\h{T,y)\ + \hy{r,y)\\dy+ r I \ex{T,s,y)\dyd 



(5.26) 



<i^(l+||/i(r,-)||cHR")) + 



t JR" {s — t) 



_\\^-v\± 

n+l 

-\^x{T,s,y)\dyds. 
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The above can be iterated as follows: 

4(t,-t) 

\e,{T,s,y)\dyds 

i" (s-t)^~ 



<i^(l + ||Mr,-)||cHR-))+ / / -^K{l+\\h{T,-)\\c^in-))dyds 

Jt JR" — t) 2 



Ke~ f f Ke~ 

-\ea:{T,r,z)\dzdrdyds (527) 



/t ^R" (s — i) 2 ^R" (r — s) 

<K{l+\\h{T,-)\\c^iRn)) 



_i" ^ Jt Jr" {s — t) ^ (r — s) 

i-T 



Xe~ -K-—) \ 
i±r 1 Ts+r dydsj\idx{T,r,z)\dzdr 

2 It* — Ql 2 ^ 



<i^(l + llMT,-)llcHR"))+ / / \Q.iT,r,z)\dzdr. 

Jt JR" (,r — tj 2 

In the above, we have used Lemma 3 of Chapter 1 in [9]. We can repeat the above procedure 2n times and 
then use Gronwall's inequality to obtain 

\Q,{T,t,x)\<K{l + \\h{T,-)\\c^Rn^), V{t,x)e[T,T]xn". (5.28) 

Now, we let v'{-,-) € C([0,T];Ci(]R")), i = 0,1. Let Q'{t,-,-) be the corresponding solutions of (5.20). 
Then 



e\T,t,x)-e°{r,t,x)= [ [ r°{t,x;s,y)[{b{s,y,vl{s,y)),el{T,s,y)-el{T,s,y)) 



t JR" 



+ {b{s,y,vlis,y))-h{s,y,v%s,y)),el (r, ,s, y) ) 

+g{T, s, y, vl{s, y)) - g{T, s, y, v°{s, y))] dyds, {t, x) G [r, T] x R" 



(5.29) 



and 



el{T,t,x)-el{T,t,x)= [ [ T°it,x;s,y)[{el{T,s,y) - &'i{T,s,y),b{s,y,vl{s,y))) 

Jt Jr" 

+ ( 0° (■^: s, y),b{s, y, u^(s, y)) - b{s, y, v°{s, y)) ) 

+g{T, s, y, vl{s, y)) - g{T, s, y, v°{s, y))] dyds, {t, x) € [t, T]xR", 

Hence, 



(5.30) 



A|x-y|^ 

nfi^p 4(s-t) / 
(\el(r,s,y)-e'',iT,s,y)\ 
i," (s - t) 2 ^ 

+ {\@l{T,s,y)\ + 1) \vl{s,y)-v°{s,y)\)dyds 



'' Ke 



(5.31) 

Jt Jr^ (s — t) 2 

+K{T - t)^l + Whir, OllcHR")) hli- > •) - . ■)\\c([r,T]xRn). 
Then, similar to (5.28) obtained from (5.26) via (5.27), we can obtain 

\el{r,t,x) - el{T,t, x)\ < K{T - t)^ (1 + \\h{T, OllcMR")) Hi' , ') - <{' . •)IIc([.,t]xR"). (5.32) 
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On the other hand, from (5.29), we have 



\@\T,t,x)-@\T,t,x)\< / -^—^(\Ql(T,s,y)-Ql{T,s,y)\ 

+ {\@l{T,s,y)\ + l)\vl{s,y)-vl{s,y)\) dyds. 
Comparing the above with (5.31), we see that the following must be true: 

\Q\T,t,x) - e°(r,t,x)| < K{T -t){\+ \\h{T, OllcHR")) ll^x(- , - vl{- , ■)\\car,T]xH-y 
Hence, we obtain 

||ei(r,.,.)-e°(r,-,-)||c([x.T];CW) 

< KiT - t)i{l + WHt, OIIchR")) . •) - ^°(- > ■)llc(KT];CHR"))- 



(5.33) 



(5.34) 



(5.35) 



From the above procedure, we see that K > in the above is an absolute constant, independent of (r, t) e 
D[0,T]. Hence, in particular, we have (denoting V^{t,x) = Q^{t,t,x)) 



ll^^(- ^ ■)\\c([T-5,T];CHri")) 

< KSi{l + \\hi-, ■)\\b{[o,t];CHR'^))) W^^i' - ") " "°(- ' ■)\\c{[T-S^T]:CHR"))- 



(5.36) 



Clearly, by choosing S > small, we get a contraction mapping v{- , •) V{- , •) on C{[T — S,T]; C^(]R")). 
Therefore, this map admits a unique fixed point. Since we may obtain similar estimates on [T — 2d,T — 6]), 
etc., one sees that the fixed point will exists on the whole space C([0, T; C^(R")) for the map v{- , •) V{- , •). 
Then we obtain the well-posedness of the following: 



e(r, t,x)= [ rO(t, x; T, y)h{T, y)dy + [ [ r°(i, x; s,y)[{ b{s, y, 9, 

JR" Jt JR" 



is,s,y)),exiT,s,y)) 



(5.37) 



+g{T, s, y, e^(s, s, y))] dyds, (r, t, x) G D[0, T] x R", 



Finally, by the regularity of the above expression, we know that 0(r, t, a;) is C^+" in x, in t for some 

a e (0, 1), and the PDE (5.8) is satisfied. □ 

Recall that in Section 4, by assuming the convergence of 0'^(- , ■ , ■) and V^{- , •), we get the equilibrium 
HJB equation for 6(- , • , •) and then V{- , •) is characterized by an equilibrium HJB integral equation. We 
now want to show that under conditions ensuring (P), we do have the expected convergence. This will 
make our whole procedure satisfactorily complete for certain cases, at least. For the sake of simplicity, we 
assume that all the involved functions are bounded and continuously differentiable up to a needed order with 
bounded derivatives. 

When (5.6) holds, for fc = 0, 1, • ■ • , — 1, we have 



e^{T, t, x) + tr [a{t, a;)el(r, t, x)] + { b{L x, VX^"(t), t, x, V^it, x))) , S^ir, t, x) ) 

+g^{T,t,x,^j{e^{t),t,x,v^{t,x))) =0, {t,x) e [ife,r] xR", 

e^{T,T,x) =h^{T,x), xeR", 



(5.38) 



with 



N-l 



h^{T,x) = J2 Ktk-ux)Iit,_,,t,)ir), {t,x) e [0,T] X R", 



fe=i 



JV-l 



g^{T,s,x,u) = g{tk-i,s,x,u)I[t^_^^tk){T-), {t,s,x,u) G D[0,T] x R" x JJ. 
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k=l 



+ {b{s, y, ip{i^{s), s, y, V^{s, y))) - b{s, y, ipis, s, y, y))) , e^iCr, s, y) ) 
s, y, ipii^is), s, X, V^{s, y))) - g(r, s, y, ■i/'(s, x, V^{s, y)))] dyds. 

Thus, 



/■ _^g~ 4(T-t) 

|en(r,t,x)-e4T,i,x)l < / \h]J{T,y)-hy{T,y)\dy 



Since (5.38) is well-posed, we have 

V^{t, x) = Q^{£^{t), t, x), {t, x) G [0, tN-i) X R". (5.39) 
Also, by the assumed uniform Lipschitz continuity of r [h{T,x),hy{T,y),g{T,t,x,u)), we have 

\h^{T,x) - h{T,x)\ + \h^{T,x) - h^{T,x)\ + \g^[T,s,x,u)^g[T,s,x,u)\ < K\\n\\, 

V(t, s, X, u) e Li[0, TjxR" xU. 

Next, by Proposition 5.1, we have 

en(T, t, x) - e(r, t, a;) = / ?«(*, x; T, y) [/^"(r, y) - /i(t, y)] rfy 

+ / / rO(i,x;s,y)[(6(s,y,^(£n(s),s,y,yn(s,y))),en(r,s,y)) 

JR" 

- ( b{s, y, tp{s, s, y, V^s, y))) , Q^ir, s, y) ) 

+9^{r, s, y, i){i^{s),x, V^{s, y))) - g{T, s, y, ip{s, x, V;(s, y)))] dyds 
= [ r°{t, x; T, y) [h^{T, y) - h{r, y)] dy 

JR" 

+ / / rO(t,a;;s,y)[(6(s,y,^(^n(s),s,y,yn(s,y))),en(r,s,y)-e,(T,s,y)) 
Jt Jr" 

+ {b{s, y, ^-(^"(5), .s, y, ^^(.s, y))) - b{s, y, ^{s, s, y, V^{s, y))) , e,(T, s, y) ) 
+5"(t, s, y, il){i^{s), X, 14"(s, y))) - g{T, s, y, tp{s, x, 14(s, y)))]dyds. 

Consequently, 

en(T,t,x)-e,(T,t,x) = / T"{t,x;T,y)[h]jiT,y)-hy{T,y)]dy 

Jr" 

- [ T°{t,x;T,y)p{t,x,T,y)[h^{T,y) - h{T,y)]dy 

rTr (5.42) 

+ / / T'iit,x;s,y)[{b{s,y,ij{f^{s),s,y,V^is,y))),e^ir,s,y)-&4T,s,y)) 



+ 11 1' ^ (|eg(T,a,y) - e.(T,a,y)| + [^"(s) - a| + |lf (5,y) - T4(a,y)| 
Jf Jr" (r-i)-2- V (5.43) 

+ 15" (t, s, y, V'('^> s, X, 14(s, y))) - g{T, s, y, ip{s, x, 14(s, y))) |) c^yc^s 

A|x-i/|^ X|x-h|^ 

f Ke MT-t) r 4(T-t) / \x-y\'^\ 



nt-p- 4(T-t) / N 
— —^(\e^{r,s,y) - e.{T,s,y)\ + \\U\\ + \Q^{£^{s),s,y) - e,{s,s,y)\]dyds. 



It Jr" {T-t) — 
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Therefore, 



sup \e^iT,t,x)-e,{T,t,x)\<K\\n\\+ / \Q^{i^{s),s,x) -Q4s,s,y)\dyds 

■e[o,t] Jt -/R" (T-t)~ 



re[o,t] Jt Jw [T-t] 



(5.44) 



It (T-t)^ re[o. 

This yields that 



f f Ke~ MT-t) 

+ -7Z, 7^ sup |e^(T,s,t/) -e^(T,s,y)|dyds. 

Jt JR" (T-t)-2- refO.sl 



_ >|x-k|^ 



||n||+/ / \Q^{t^{s),s,x)-e,{s,s,y)\dyds). (5.45) 

Jt Jr" (T-f)— ^ 

Likewise, we can have a similar estimate for |0^(t, a;) — ©(r, t, a;)|. Hence, we obtain 



sup ||en(r,t,-)-e(r,t,.)||ci(R") 

(r,t)e£>[0,T] 



(5.46) 



< K 

Prom this, our expected convergence follows. 



(l|nll+/ / -——-^\Q.{t''{s),s,x)-@^{s,s,y)\dyds). 



6 Some Special Cases 

In this section, we are going to look at several important special cases. We will mainly look at the corre- 
sponding forms of our equilibrium HJB equations. 

6.1 A Linear-Quadratic Problem 

Let us look at the LQ problem. For any initial pair (i,x) G [0,T) x R", the state equation is 



dX{s) = [A{s)X{s) + B{s)u{s)]ds + [Ai{s)X{s) + Bi{s)u{s)]dW{s), s G [t,T], 
X{t) = X, 



(6.1) 



with the cost functional 



Jit, x; «(•)) = \^t[J^ ( ( Q{t, s)X{s), X{s) ) + ( R{t, s)u{s),u{s) ))ds+{ G{t)X{T), X{T) ) ] . (6-2) 



Then 



1 r T 
H(r, t,x,u,p,P) = {p,A{t)x + B{t)u) + -tr |^ [Ax {t)x + {t)u) [Ax {t)x + Bi {t)u) P 



{ Q{t, t)x, x) + { R{t, t)u, u ) 



= {p,A(t)x)+]^ { [Ax{tfPAx{t)+Q{T,t)]x,x) 

+1 ( [R{t, t) + Bx {tfPBx (t)] u,u) + {u, B{tfp + Bx {tfPAx {t)x ) . 

This yields 

V(r, t, X, p,P) = - [R{t, t) + Bx [tfPBx (t)] [B{tfp + Bx {tfPAx {t)x] , 
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and 

H(r, t, X, ip{t, t, X, p, P),p, P) 

= {p,A{t)x)+^{[A, {tfPA, {t) + Q{T,t)]x,x) 

+i ( [R{t, t) + Bi {tfPBi {t)] ^{t, t, X, p, P) , tPit, t, X, p, P) ) 
+ ( V(t, t, X, p, P) , B{tfp + Bi {tfPAi {t)x ) 
= {p, Ait)x )+l{[Ai (tfPAiit) + QiT, t)] X, X ) 

+ 1 ( [R{T,t) + Bi{tfPBi{t)] [R{t,t)+Bi{tfPBi{t)y\B{tfp + Bi(tfPAiit)x], 

[R{t, t) + Si {tfPBi {t)] [B{tfp + Bi {tfPAi {t)x\ ) 
- { [R{t, t) + Bi{tfPBi{t)] [Bitfp + Bi{tfPA,{t)x] , B{tfp + B^{tf PA^{t)x) . 
Hence, the equilibrium HJB equation takes the fohowing form: 

(r, t,x) + { e^(T, t, x), A{t)x ) +^ ( [Ai(i)^e^^(r, t, [t) + Q{t, t)] x, x ) 

+ 1 ( [Rir, t) + Bi {tfe^^ir, t, x)Bi {t)] [R{t, t) + B^ {tfe^t, t, x)Bi (t)] 
• [B{tfe, {t, t, x) + Bi {tfe.^it, t, x)Ai {t)x] , 

[R{t, t) + Bi (tfQ,, (t, t, x)Bi {t)] [B{tfQ, (t, t, x) + Bi {tfe^t, t, x)Ai {t)x] ) ^^'^^ 
- { [R{t, t) + Bi {tfe^^it, t, x)Bi{t)] [B{tfQ^{t, t, x) + Bi {tfe^t, t, x)Ai {t)x] , 
B{tfe^{T,t,x) + Bi{tfe^^{T,t,x)Ai{t)x) = 0, {T,t,x) e D[0,T] x R", 

^ Q{t,T,x) = i {GiT)x,x), {t,x) e [o,r] X R". 

Although having a little bit complicated looking, the above has a quadratic structure which can help us to 
study the well-posedness of it. To see that, let 

e{T,t,x) = ^{P{T,t)x,x), {T,t,x) e D[0,T]xW, (6.4) 

with some undetermined map P : £>[0,T] <S". Plugging the above into (6.3), we see that the map P{- , •) 
should satisfy the following equation: 

= Pt{T,t) + P{T,t)A{t) + A{tfP{T,t) + AlitfPiT, t)Alit) + Q{T,t) 

+ [P{t, t)B{t) + A^{tfP{t, t)B^{t)][R{t, t) + B^{tYP{t, t)B,it)]-^ 

■[R{t, t)+Bi{tfP{T, t)B, (t)] [R{t, t)+B, {tfP{t, t)Bi {t)]-^[B{tfP{t, t)+B^{tfP{t, t)A^ (i)] 

-[P{t,t)B{t)+A^{tfP{t,t)Br{t)][R{t,t)+B^{tfP{^^^^ 

-[P{T,t)B{t) + Ai{t)'^ P{T,t)B^{t)][R{t,t) + Bi{t)^ 

with the terminal condition 

P{t,T) = G{t). 

Note that if P(t, t) and R{t, t) are replaced by P(t, t) and i?(r, <), respectively, the above becomes a standard 
Riccati equation with a parameter r. The appearance of P{t, t) and R{t, t) makes the above non-standard. 
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We may rewrite the above as follows (suppressing t in P(r, t), etc., for simplicity): 

= Pt(r) + P(t)A + A^Pir) + A[P(r)Ai + Q(t) + [P{t)B + P(t)Bi] [P(i) + Pf P(i)-Bi] 
• [P(r) + Pf P(t)Pi] [R{t) + BlP{t)B,] [B^ P{t) + Pf P(t) Ai] 

- [P{t)B + P(i)Pi] [P(t) + BlP{t)B,] [P^P(t) + PfPWAi] 

- [P(t)P + Af P(t)Pi] [P(t) + BfP{t)B,] [B^P{t) + BfP{t)A,] 
= Ptir) + P(r)(A -B[Rit) + Pf P(t)Pi] [P^P(t) + Pf P(t)Ai]) 

+ (a - P [P(i) + P?^P(i)Pi] [P^P(t) + Pf P(t) Ai] ) ^P(r) + Q(r) 

+ (Ai - Pi [P(t) + Pf P(t)Pi] [P^P(i) + Pf PCOAijj'^PM 

• (Ai - Pi [P(t) + Pf P(t)Pi] [P^P(t) + P(t) Ai] ) 

+ [P{t)B+A'^P{t)Bi\ [R{t)+B'[P{t)Bi\ "'p(t) [P(t)+Pf P(t)Pi] ~\b'^ P{t)+B'[ P{t)Ai\ . 

Denote 

r(t) = [R{t, t) + Bi{tfP{t. t)Bi (t)] [B{tfP{t, t) + Bi{tfP{t, t)Ai {t)] , 

= -P(t)r(t), Ai{t) = Ai{t) - Bi{t)r{t), {T,t) e D[0,T]. (6.5) 

O(T,i) = Q(T,t) + r(t)^p(T,t)r(0, 



Then the equation for P(- , •) can be written as follows: 

f Pt{T,t) + P{T,t)A{t) + A{tfP{T,t) + Mtfp{T,t)Mt) + Q{T,t) = 0, (T,i) e P[0,T], 

1 P(T,T) = G(r), Te[0,T]. 
Next, let $(• , •) be the fundamental matrix of (A{-), Ai{-)) , i.e., the following holds: 

f d^s,t) = A{s)^s,t)ds + Ai{s)^s,t)dW{s), sG[t,T], 
I $(t,i) = ^- 

Applying Ito's formula to s i->^ ( P(r, s)$(s, f)a;, $(s, t)a; ) on [t, T], we have 

( $(T, i)^G(r)$(T, t)a;, a; ) - ( P(t, i)a;, ^) = - ( *)^' ) 

+ ^ ( [P(r, s)Ms) + Ai{sfP{T, s)] $(s, t)x, $(s, <)a; ) dW^(s), 



(6.6) 



(6.7) 



which leads to 

It 



P{t, t) = $(r, t)^G(r)$(T, t)+ ^(s, ^)^Q(t, s)$(s, t)ds 

$(s,t)^[P(r,s)li(s)+li(s)^P(r,s)]$(s,t)dW(s), (T,t) G P[0,T]. 



(6.8) 



Note that although P{T,t) is a deterministic function, the above representation is stochastic. From the 
above, we have 

P(T,t)=Et[$(T,i)^G(r)$(T,i)+^ ^{8,tf[Q{T,s)+T{sfR{T,s)T{s)\^{s,t)ds^, {T,t) e P[0,T]. (6.9) 
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In particular, taking t = t and denoting P{t) = P{t,t), one has 

P{t) = -Et[<^{T,tfG{t)<P{T,t)+j^ <^{s,tf[Q{t,s)+r{sfR{t,s)r{s)]<P{s,t)ds\, te[Q,T]. (6-10) 
Combining the above, we end up with the following system for the function P{-): 

P{t) = Et[HT,tfG{t)^T,t)+J^ 'i>{s,tf[Q{t,s)+r{sfR{t,s)r{s)]^s,t)ds\, t G [0,T], 

^{s,t) = I+j [A{r)-B{r)r{r)]<^{r,t)dr+J [Ai{r)-Bi{r)r{r)]^{r,t)dW{r), {t,s) e D[0,T], ^^'^^^ 

J{t)= [R{t,t) + B,{tfP{t)B,{t)]~\B{tfP{t) + Bi{tfP{t)A,{t)], t€[Q,T]. 

Wc refer to the above as a RiccaM- Volterra integral equation system for the corresponding (time-inconsistent) 
LQ problem. Note that the above is actually a coupled forward-backward stochastic Volterra integral equa- 
tion system (FBSVIE, for short). Some relevant results concerning backward stochastic Volterra integral 
equations (BSVIEs, for short) can be found in [25, 26]. If ($(• , ■), -P(-)) is a solution to the above, then the 
time-consistent equilibrium control is given by 

u{t) = -T{t)X{t), iG[0,T]. (6.12) 

In the case that 

^i(-) = 0, Bi(-)=0, (6.13) 

the above (6.11) is reduced to the following Riccati- Volterra integral equation system (for a deterministic 
time-inconsistent LQ problem): 

P{t) = ^{T,tfG{t)^{T,t) + j^ ^{8,tf [Q{t,8) + T{sf R{t,8)T{s)]^{s,t)ds, tG [0,r], 
$(s, t) = I + f [Mr) - B{r)T{r)] $(r, t)dr, {t, s) G D[0, T], ^^'^^^ 

I r(t) = Rit, t)-^B{tfp{t), t e [0, T], 

and the time-consistent equilibrium control is given by (6.12) with a simpler r(-). This recovers the case 
presented in [27] where the well-posedness of (6.14) was established. 

For (6.11), we have the following result. 

Proposition 6.1. Suppose 



A{-),A,{:) € C([0,T];]R"><"), S(-),i?i(-) G C([0, T]; ]R"><"^), 

g(-) G C{D[Q,T];Sf), R{-) G C{D[Q,T];S!^), G(-) G C{[0,T];SI). 



(6.15) 



Further, suppose 



sup 
Pesi 



[R{t,t) + Bi{tfPBi{t)] \B{tfP + Bi{tfPAi{t)] =L<oo. (6.16) 



Then (6.11) admits a unique solution. 

Proof. Let A'[r, T] = C([t,T];S!^) which is a complete metric space with the metric induced by the 
norm in C([r,T];5"). For any p(-) G X[0,T], define 

T{t;p{-)) = [R{t,t) + B,{tfp{t)B,{t)]-\B{tfp{t) + B,{tfp{t)A^{t)], t G [0,T]. 
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By (6.16), we have 

\r{t:p{-))\<K, te[o,r], 

with the bound independent of p{-) G X. Let $(• , •) = $(• , • ;p(-)) be the solution to the following: 



$(s, t)=I + J [A{r) - B{r)T{r; p(-))] t)dr 

+ [A,{r)-B,{r)r{r;p{-))]^r,t)dW{r), {t,s) € D[0,T]. 



We have 



E 



sup |$(s,i)|2 

(t,s)eD[0,T] 



for some absolute constant K. Next, we define 

P{t) = P{t;p{-)) = Et[^{T,tfG{tMT,t) 

+ ^s,tf[Q{t,s)+T{s;p{-)fR{t,s)T{s;pi-))]<^>{s,t)ds, t€[0,T]. 

Clearly, P(-) G A'[0,T]. We want to show that p{-) i->- P{- ;p{-)) admits a unique fixed point. To this end, 
for any p^{-),p'^{-) G A'[0,r], we denote 

P(.) = r(.;p'(.)), $'(•,•) = $(-,-;p'(-)), P\-) = P{- i = l,2. 

Then (suppressing t) 

\THt)-r\t)\ = \{R + BfpiBiy\B^pi + Bfp.Ai) - {R + Bfp2B,)-\B^p2 + B^p2A,)\ 
<\{R + BfpiBi)-^ I \B^{pi - p2) + Bfipi - p2)Ai I 

+ 1 [{R + BIp^B^)-^ ^iR + B^p2Bi)-'] {B^p2 + Bfp2-Bi)| 

< K\px-p2\ + \{R + BfpiBi)-^Bf{pi -p2)Bi{R + Bfp2Bi)-\B^p2 + Bfp2Ai)\ 

< K\pi{t) - P2{t)\. 



Next, 



$i(s,t) - ^\s,t) = y {A{r)[^\r,t) - ^\r,t)] - B{r)[r\r) - T\r)]^\r,t) 

-B{r)T^{r)[^\r,t) - ^^{r,t)]}dr 
+ f {M (r) [<5i (r, t) - $2 (r, t)] - B, (r) [T\r) - (r)] $i (r, t) 
-Bi{r)r^{r) [$i(r, t) - <^^{r, t)] }dW{r). 



Then 



E 



sup \^\s,t)-^^{s,t)f 
se[t,T] 



<K f \T^{r)-V^{r)\^dr<K f \p^{r) - p'^{r) 
Jt Jt 



'dr. 
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Consequently, 

\P\t) - P^{t)\ < E{\^\T,tfG{t)^\T,t) - $2(T,t)G(t)$2(T,t)| 

I $1 (s, i) [Q{t, s) + s)T' (s)] $^ (r, t) 

t) [Q{t, s) + T\s)R{t, s)r\s)] $2(r, t)\dr} 
i^E{(|$i(T,t)| + |$2(T,i)|)|$i(T,i) - $2(T,i)| 



< 



+ 



+ 



< K[j^ \p\s)-p\s)\''ds + [[f^\p\r)-p\r)\^drf 

+ {m\s,t) - + \T\s) - T\s)\^f\ds] < K(^j^ \p\s)-p\s)\^dsy , 

with K >Q being an absolute constant. Hence, we obtain 

max \P^{t)-P'^it)\<K{T-T)^ max \p^ {t) - p"^ {t)\. 

Hence, the map p(-) ^ P(-) is contractive on X[t,T\ as long as T — r > small. Then a usual argument 
applies to obtain a unique fixed point of p(-) i->- P(-) on A'[0, T\. This proves the well-posedness of (6.11). 



To conclude this section, let us make a remark on the condition (6.16). It is not hard to show that when 
m = n and Bi{-)~^ exists and bounded, then (6.16) holds. Apparently, this is a restrictive condition. We 
hope that in our future publications, such a condition can be removed. 

6.2 A generalized Merton's problem 

In this subsection, let us look at the Merton's portfolio problem with general discounting. The results in 
this subsection is comparable with some of the results in [7] and [18] . Let us recall 



and 



Then 



dX{s)= [rX{s) + {^-r)u{s)-c{s)\ds + (Tu{s)dW{s), sG [t,T], 
X{t) = X, 



J{t, x; u{-), c(-)) = E J / i/(t, s)c{sfds + p{t)X{Tf 



lH.{t,s,x,u,c,p,P) = p[rx + (/X — r)u — c] + -a^u^P + v{t,s)c^ 



a'P 



rxp - 



,2p 



+ v{t, s)c^ — pc. 



The maximum of (u, c) ]H(t, s, x, u, c,p, P) is attained at 

(m - r)p 



I3v{t, s)\ih 



(j'^P ' 
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(6.17) 



(6.18) 



We denote 



Then 



m t, x,p, P) ^ in, C-) = ( - fcl^, [^] ^) . 



and 



H(r, t,x,'tlj{t,t,x, p, P) , p, P) 



rxp 



= rxp + 



,4p2 



a2P cr2p 



{^-rfpPfp v\ ^ [Pv{t,t)]- 



2a^P 



[u{T,t)p - piy{t,t)p\. 



Hence, the equihbrium HJB equation reads 



(yU - r)'^Qai{t,t,x)(da:x{T,t,x) ( Qx{t,t,x) &x{tJ- 



2a^e^4t,t,x) 



<dxxit,t,x) Oxx{T,t,x) 



Qt (t, t,x)+ rxQx (t, t, x) + 

p 

+ [^^(^'^)]^ [v{T,t)ex{t,t,x) - ^v{t,t)Qx{T,t,x)\ =0, (T,i,a;) e r>[0,T) X (0,oo), (6.19) 

Q^{t,t,x)T^ 

e(T,i,o) = o, 

[ Q{t,T,x) = p{t)x^ , 



{T,t) & D[Q,T], 
(t,x) e (0,oo) X [0, oo). 



Let 
Then 



e(T, i, a;) = ip{T, t)x^, (r, t, x) G £)[0, T] x R". 



= v.t(r, i) + r/3¥)(r,t) + 
= (/Jt(r,t)+r/3(/j(T,t)4 

= (^((T,t)+/3 



2a2(l - /3) 



2o- (1 -P) ^{t,t)^-e 



and 

with A given by (2.11). Thus, 



<p(t,i)' 
<^(r,T) = p(T), 



it ^<^(s,s)^ 
{T,t) & D[Q,T]. 

Hence, we obtain the following integral equation for 1 1-> y>(t, t): 
^{t,t) = e^^^-'^-' ^^)"^ p{t) 

+ [\'^^-^^-^i:i^)'^'^^'(!^)^uit,s)ds, i e [o,T]. 
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(6.20) 



Once such an integral equation admits a unique solution t i-> ip{t,t), we will obtain the time-consistent 
equilibrium value function 

V{t,x) = Q{t,t,x) =ifi{t,t)x'^, {t,x) e [0,T] X [0,cx)), 

and the time-consistent equilibrium control 



To establish the well-posedness of (6.20), we set 



(6.21) 



Then (6.20) is equivalent to the following: 



zit) = e"(^-*)-'^/^('^)^'^V(i) + j[^e"^'"*^"''-^'''^^^^''^(T)^^^(i,r)rfr, t G [0,T]. (6.22) 

Wc have the following result. 

Proposition 6.2. Let v : -D[0,T] -> (0,/3] and p : [0,T] (0,cx)) be continuous, and 



^ -\VLv{t,S) 

A = sup < 00. 

0<t<s<T s — t 

Suppose z : [0, T] (0, oo) is a solution to (6.22). TJien 

te[o,T]^ ^ ~ te[o,T]^ ' ^ ' 

Proof. We may write (6.22) as follows: 



(6.23) 



(6.24) 



z{t)e 



-A(T-t)+/3 



Jt =p\t)-\- / v(t,T)z(T) 1^-^ Z[T)e ' '^Jt 



dr. 



Denoting 



we have 



Note that 



z{t) = z{t)t 



-X{T-t)+l3 z{s)'^ds 



tG[0,T], 



r-T 

z{t) = p{t) + j v{t,T)z{T)^z{T)dT, te[o,r]. 



po = min pit) < pit) < pi = max pit), < vit,T) < j3. 
te[o,T] te[o,T] 



(6.25) 
(6.26) 



Thus, (6.25) implies 



/I 
z{T)^z{T)dT, tG[0,T]. 

Then by Gronwall's inequality, we obtain 
which leads to 

^(t)<Pie^(^-*), te[0,T]. 
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Next, (6.23) implies 

Ht,s) >e-'^^'-'\ {t,s) € D[0,T]. 
Consequently, we obtain from (6.25) that 

e-^^^-'h{T)^z{T)dT, t€[0,T], 

which is equivalent to the following: 

z{t)p^ [z{T)e-^'']dT = Cit). 
C'{t) = -Apoe-^* - z{t)^ [z{t)e-~^*] < -Apoe"^* - ^(i)^C(*), 



(6.27) 



(6.28) 



Then 
which yields 



C(i)e- 



Hence, 



Then 



poe-^^ - C{t)e 



rT 1 



< -Xpo / e-^^e 



-AT 



Apo / e-^^e 



(=> J T 



dT 



>Poe(^-^)(^-*), te[0,T]. 



This proves our proposition. □ 

Having the above proposition, wc then can easily obtain the well-posedness of (6.20) by means of contrac- 
tion mapping theorem (giving the local solvability) and a usual continuation argument. Then time-consistent 
equilibrium control can be constructed by (6.21). 

Note that by multiplying a constant to the payoff functional, if necessary, we can always make 

max i/(r, t) < B. 

(r,t)eD[0,T] 

On the other hand, in the case that s i-> u{s, t) is differentiable, we have 

lu //(/,. S) Us{t,t) 



lim ■ 
sit s — t 



y{t,t) 



Thus, condition (6.23) is ensured by the boundedness of the right hand side of the above, which is not too 
restrictive. 
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7 Appendix 

In this appendix, we present some detailed calculations. 

Example 2.1. Recall that we are considering the following one-dimensional controlled linear SDE: 

f dX{s) = u{s)ds + aX{s)dW{s), s € [t, T], 
I X{t) = X, 

with cost functional ^ 

J{t,x;u{-))=-Et[J^ \u{s)fds-{-g{t)\X{T)\^], (7.2) 

where u > is a constant and g{t) is a deterministic non-constant, continuous and positive function. For 
such a linear-quadratic optimal control problem on [t, T] (with deterministic coefficients) and with t £ [0, T) 
fixed, the Riccati equation takes the following form: (note that t £ [0, T) is a parameter) 

I Ps{s,t)=P{s,tf -a^P{s,t), se[i,T], 
1 P{T,t)=g{t). 

Let us solve the above Riccati equation. By separation of variables, we have 

, dP _ 1 / 1 l\.p 

Integrating from s to T, one has 

'9{t)-^\ .fP{'%t)-a\ ,(9{t)-a^ P{s,t) 



^ ' V g{t) J \ P{s,t) J V g{t) P{s,t)-a^J 



Then 



Pis,tMt) - a^] 
[P{s,t) - a^]g{t) 

Hence, 

^'•'''° .'+»(V)(e^->r-.,-l) - 
and the optimal control is given by 

u{s) = -P{s,t)X{s), se[t,T]. 
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Thus, the closed- loop SDE reads 



J dX{s) = -P{s,t)X{s)ds + aX{s)dW{s), s G [t,T], 
\ X{t) = X. 

Consequently, the optimal state process is given by 



a2+5(i)(e-'(r-*) -1) 



since 



-/'p(M)* = -f- 



-W{t)] 



X, se[t,T]. 



2+5(i)(e'^=(r-0)-l) 



de 



d\cj^ + .gmre- (^-») - 1)1 + ^(t)(e^^(T-.) _ 

— In ' 



t (T2+5(t)(e-'(2^-e)-l) 



+ 5(t)(e'^'(^-*) - 1) 



)■ 



Consequently, the optimal control also admits the following open-loop form: 

a'^g{f)e''\T-8)^~^(8~t)+a[W{s)-W{t)]^ 



u{s) = u{s\ t, x) = —P{s, t)X{s) 



Recall that 



,klW{s)-W{t)] 



c72+g(f)(e-=^(^-*)-l) 



S>t>0. 



In fact, for fixed t, applying Ito's formula to s i->- ip{s) = e'^I'^^*^ ^W] ^ ^ffQ have 



dip{s) = k(p{s)dW{s) + —(p{s)ds, s >t, 



^(t) = 1. 



Hence, 



d{Et[ipis)]} = —Etiifis)], s>t, 



^ m^{t)] = 1, 

which leads to what we want. We can check that 

J(i,a;;M(-))=EJ / \u{s)fds + g{t)\X{T)\' 



a2g(t)e'-'(^-')x2 
c72+5(t)(e^^(T-f) -1) 



= P{t,t)x^. 



Next, let T G {t,T). We consider the corresponding LQ problem starting from the initial pair (r, X(r)). 
Then the optimal state process, denoted by X{-), must be given by 



X{s) 



1) 



(«-)+-[^w-^m1x(t), se[T,T], 



C72+5(T)(e<^'(^-^) -1) 

and the optimal control, denoted by u(-), should be given by 

u{s) = -P{s,r)X{s) = , ^ , s G [r,T]. 



Note that 



J(r,X(r);S(.)) = P(r,r)|X(r)|2 = 



ct2 + g(^)(e^^(T-r) _ 1) 

a2g(r)e-'(^-) 



(T2+g(T)(e<^'(^— ) - 1) 
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On the other hand, 



Thus, for s e [r,T], 



X{7 



C72 + g{t){e^'i'r-t) _ i) 



T[W{T)-W(t)] 



X. 



(t2 + ff(i)(e'^'(r-*) - 1) 
(72 + 5(t)(e-^(T-r) _ 1) ^ 



and 



u{s) = -P{s,t)X{s) 



-4(s-r)+<7[VV(s)-Vy(r)]^(^^_ 



(72 + .g(t)(e'^'(r-«) - 1) f72 + 5(i)(e^'(^-^) - 1) 



Then 



J(r,X(r);«(-))=E, 



(a)rd5 + 5(r)|X(r)p 



^ a^£,(t)2e2<^'(r-«)-'T'(s-r) + 2ff[W(s)-W^(r)]|^(^)|2 

[cr2+5(t)(e°-'(^-^)-l)]2 
a4g(r)e-"'(^-")+2'^[^(^)^'^(")l|X(r)|2 
^ [a2 + (/(i)(e-'(r— )-l)]2 

a^5(i)2|X(r)|2 



:a2(T-r) _ 1)]2 + 



[(T2 + 5(t)(e 

a25(f)2|X(r)pe^'^'^ -"J(e'^'^'^ - 1) + a''g{T)e 



[a2 + g{t){e^'(T-T) _ i)]2 



[cT2+5(t)(e-'(^--)-l)]2 
[5(f)2(e-'(^-) - 1) +a25(r)]a2e-^(^--)|X(r)|2 



[CT2 + (/(t)(e<^^(r-^)-l)]2 

Consequently, the following holds: 



J{T,X{T);u{-))-JiT,X{T);u{-)) 



5(r) 



git ne-'(^-^^-l) + a^g{r) 

2 + g{t)ie^'(T-T) _ i)]2 ^2 + 5(r)(e^^(r-^) - 1) 

~ [(t2 + g{t){e'''(T-r) _ 1)]2[^2 + 5(r)(e-^(T-r) _ 1)] 

_ a4(e<^^(T-r) _ _g(^)]2ga^(r-r)-^^(r-t)+2.[M/(r)-l^(0]^2 

~ [a2 + (7(t)(e-'('^-*) - 1)]2[ct2 + g(r)(e^'(^-^) - 1)] ' 

which is strictly positive unless x = 0, or g{T) = g{t). This means that the problem is time-inconsistent. 
Exeimple 2.2. (Generalized Merton's portfolio problem) Recall the following controlled SDE: 



f dX{s) = [rX{s) + (m - r)u{s) - c{s)] ds + au{s)dW{s), s G [t, T], 
I Xit) = X, 

with payoff functional 

J{t,x;u{-),c{-))=Et 



J v{t, s)c{sfds + p{t)X{T; t, x, u{-), c(-))^ 
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(7.3) 



(7.4) 



where , •) and p{-) are given positive-valued functions, and 13 £ (0, 1). As a convention, we define 

a; < 0. 



-00, 



The optimal control problem is to find a pair (u(-), c(-)) such that J{t, x; u{-), c(-)) is maximized. To approach 
this problem, we use dynamic programming method. More precisely, for any (s,y) € [t,T) x [0,oo), let 



J*(s,y;«(-),c(-)) =]Et[ / i^{t,T)c{TfdT + p{t)X{T;s,y,u{-),c{-)) 

J s 

Define the value function V*{- , •) (which is parameterized by i G [0,T]) by the following: 
V\s,y)= sup J\s,y;u{-),c{-)), {s, y) € [t,T] x [0,oo). 



(7.5) 



(7.6) 



Due to the problem being a maximization problem, the above convention forces X{s) to stay nonnegative, 
in particular, the initial state a; > has to be assumed. In another word, V*{s,x) is only defined on 
[0, T] X [0, oo). By Girsanov's theorem, we know that 

W{s) = i^—!-s + aW{s), s>0 
a 

is a standard Brownian motion, with the natural filtration (augmented by all the P-null sets) coincides with 
F = {J-t}f>o- Then (7.3) is equivalent to the following: 



dX{s) = [rX{s) - c{s)] ds + au{s)dW{s), s e [t, T], 
X{t) = X, 



(7.7) 



Therefore, for any initial pair (t.x) G [0,T) x [0,oo), and a control (■«(•), c(-)), the unique solution X{-) 
X[-;t, X, u{-), c(-)) of the above is given by the following: 



X{s) = e''^'-*^x- [ e'''^'-''^c{T)dT + a [ e'''^'-''^u{T)dW{T), s € [t,T]. (7.8) 
Jt Jt 

If the initial wealth a; = 0, to keep the wealth X{-) non-negative, we have to take w(-) = c(-) = 0, leading to 

X{s) = 0, sG[t,T]. 

This means that 



v\t,o) = o, ie[o,T], 



(7.9) 



which gives the boundary condition for V*{- , •) on a; = 0. Now, let us return to (7.3)-(7.4). In the case that 
V*{- , •) is differentiable, it satisfies the following HJB equation: 



= V,'{s, y) + sup \v^{s, y) [ry + {fi - r)u - c] + ^^^^^^(s, y) + v{t, s)c^ 

(m,c) '- ^ 

= V*{s, y) + ryVlis, y) + sup - r)V^{s, y)u + \cj'^V^y{s, y)u'^] + sup [y{t, s)c^ - cV*{s, y)] . 

Assume, for the time being, that the following holds: 

V^{s,y) > 0, V^y{s,y) < 0, {s,y) G [t,T] x (0,oo). 

Then 



(7.10) 



sup 

«6R 



{n-r)V*{s,y)u+^a^V*y{s,y)u^ 
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{li-r)^V^{s,y) 



>0, 



(7.11) 
(7.12) 



with the maximum attained at 



-t, , {n-r)V^{s,y) 



and 



sup[i^{t,s)c^ -cV^{s,y)] = {1 - l3)l3T^iy{t,s)^V^{s,y)^ > 0, 



c>0 



with the maximum attained at 



Consequently, the HJB equation reads 



V^{s,y). 



> 0. 



V:{s,y)+ryV^{s,y) - ^J.y^'jl'yf + {1 - I3)l3^ iy{t,s)T^V^{s,y)^ = 0, 

(s,y) e [t,T) X (0,oo), 

y*(s,0) = 0, sG[t,T], 
_ V\T,y)=p{t)y^ ye(0,oo). 

We try to find the solution of the following form: 

V\s, y) = (^(s)/, (s, y) e [t, T] x [0, oo). 
Clearly, with such a form, (7.11) is satisfied. Then we should have 

= ^'(.)/ + r/3^(.)/ + ^^"/^ly^^^')^" + (1 - f^X^^ s)Thv{s)^y^ 
This leads to an ordinary differential equation for ip{-): 



2^2(1-/3) 



(^(s) + (l-/3Mt,s)T^(p(s)^ =0, s&[t,T], 



[ ^{T) = Pit). 



If we denote 



X = rl3 + 



2^2(1-/3) 



2a2(l - /3) 



then (7.18) becomes 



^'(s) + A^(s) + (l-/3)z/(t,s)T^V'(s)^ =0, se[t,T], 
VP(T) = p(i). 

This is a Bernoulli equation. To solve it, let 

^(s) = (^(s)T^, se[t,T]. 

Then 

/(s) = ^^^(s)T^^'(s) = -j^^is)^ (^X^is) + (1 - /3)i/(i, s)T^^(s) 



1-/3 



with 



^(s)-Ki,s)T-^, se[i,T], 
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Hence, 



Then 



J s 

-1 1-/3 

- 



Consequently, 



Therefore, the optimal control («*(•), c*(-)) for the initial pair is given by the following: 



which is not depending on the parameter t directly, and 



s€[t,T], 



c'{s) = l3^v{t,s)^V^{s,X*{s))^ =v{t,s)^ip{s)^X*{s 
v{t, s)^X\s) 



which is depending on the parameter t directly. Combining the above, we have 
sup J{t, x; u{-),c{-)) = V{t, x) = V\t, x) 

u{-),c{-) 



s€[t,T], 



T 



{t,x) G [0,T] X [0,00). 



u{t,T)c\TfdT + pit)X\TY 

Now, let i G {t,T). Then, applying the above argument, one has 

V{i,X*{t}) = [e^^'^-^p{t}^ + j\^^^-^u{i,T)^dr]'~'^X*{t}^. 



We claim that if the following is assumed 



Pit) 



dT^ 



Pit) 



dr, 



then 



(7.24) 
(7.25) 



V*{s,y) = [eT^(^-"V(i)^ + eT^(^-"V(i, r)T^drl ' V, is,y) G [t,T] x [0,oo). (7.26) 

J s -' 



(7.27) 



(7.28) 



(7.29) 



(7.30) 



(7.31) 



(7.32) 



This means that the restriction (^^(Olit t]' ^(')|[t t]) °^ ('^(')>c(-)) on [t,T] is not optimal for the initial pair 
{i,X*{t}). To prove our claim, let us denote 



r(t,s) 



z/(t, s) i-^ 



eT^(^-'^p{t)T^ + e^^^-'^iy{t,T)^dT 

e"T^(^"^V(t,s)T^ 
pit)^ + e~T^^^~^'>i/{t,T)^dT 



(7.33) 



48 



Then 



:*(s) = T{t,s)X\s), au\s) = -^—^X\s), s € [t,T], 



where is the solution to the following closed-loop system 



t X\t)=x. 



(1-/3) 



r{t,s))x'{s)ds + ^^^-^X\s)dW{s), s e [t,T], 



By denoting 



we may write the above as 



b = r + 



a = 



fj, — r 



Hence, 



Consequently, 



I rfX*(s) = [6-r(t,s)]X*(s)rfs + aX*(s)rfVF(s), se [t,T], 
\ X\t) = X. 

X^s) = x*(i)e'''-^)(^-*")-ii'r(*'^)'^^+»[^(^)-^(*")l, s G [i,T]. 

= x\f)^e^^'-^-^I*"^^*'^^''\ s e [i,T]. 



Here, we note that (recall (7.19)) 



if - rf 
2a2(l - P) 



X. 



Then 



J(t,X*(t);«(-)|j,-^],c(-)|[,-^]) = Ef[ y_ ^(i,s)c*(s)'5ds + p(i)X*(T) 
s)r(t, sfMi[X\sf]ds + p{t)Ei[X\Tf] 
y{t, s)T{t, sfe^^'-^-^ ^^'^^^"^ds + p(i)e'(^-*>'^ ^^'''^'''] X^t)^ 



By Holder's inequality, one has 



< 



< 



y_ e^^'-^v{i,s)^ds 



r{t,T)dT 



J r{t, s)e- 1^' ''^'•^^'^ds] ^ + e^(^-*")p(i)e-'^ I-^ ^^'^'^"^ 



^ A - 1 ni-^ 



where 



/■T ^ 1 ^ 1 

ai(f) = / eT^^*"*'z/(i,s)T^rfs, a2(t) = e^^'^~^ pit)"^ > 0, 
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To proceed further, we need the following elementary result. 
Lemma 7.1. Let a\,a2 > 0, 13 G (0, 1), and 

/(7) = al-V + «2"''(l-7)^ 7 6 [0,1]. 
Then 7 1— > f(j) is strictly concave and 



max 

7e[o,i]" 



Proof. Note that 



/'(7)=/3k-V-^-4-''(l-7)''-^' 



and 



/" (7) = -/3(1 - P) + 4-^(1 - -yf-^l < 0, V7 e (0, 1). 

Thus, 7 !->■ /(7) is strictly concave. By setting f'{j) = 0, we have 

ai a2 



which gives the unique maximum: 
Clearly, 



7 1-7 



7 ^ 



/ 



-) =ai" 



^ai+a2^ ^ (ai + 02)^^ ^ (ai + 02)^^ 
This proves the lemma. 

By the above lemmas, we obtain (note (7.30)) 



•^(*'^*(*);^(-)|[,-T]'S(-)|[,-T]) 

1^ e^^^-^u{t, s) (r(i, s)e- ^^^'^^'^Yds + e^^^-^ p{t)e-^ ^^''^^'''] X\t)^ 



< 



X^t)!^ = V{t,X'{t)). 



In order to have a strict inequality in the above, it suffices to have 

ai{t)^-^j{t,if +a2{t)'-^[l-j{t,i)f < [ai{t) + a2{i)]^-f' 

which is implied by 



ai{t) 



This is equivalent to the following: 



a2{t) 



ai{t)+a2{t) ■ 

eT^(^-*lp(i)T^ 
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Note that (recall (7.33)) 



- / T{t, s)ds = - [ 
Jt Jt 

-I 



ds 



In 



p{t)^ + / e T^*^ '^^iy{t,T)^dT 



In- 



Pity 



p(i)^ + /j e~'^^'^~'^^u{t,T)^dT 



Hence, we need 



P{ty 



p{t)^ + e"T^^'^"'^V(t,T)T^rfr p(i)^ + //'e"T^('^"'^V(f,T)T^dr 
which is equivalent to (7.31), proving our claim. 
Note that in the case 

u{t, s) = e-^f**-*) , p{t) = e-*(^-*) , < i < s < T, 

we have 



(7.34) 



Thus, (7.31) will not be true. When (7.34) holds, the problem is referred to as the (classical) Merton's 
portfolio problem. In this case, (with a = jE^) 



c{s,y) 



e i-f 



y_^ 

y ay 



(7.35) 



which is independent of t. This recovers the solution to the classical Merton's portfolio problem. 
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